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The  purpose  of  this  pamphlet  is  to  provide,  so^are  and  h^^ 

needed  to  conduct  the  Air  Force  Operational  Test  and  Evaluation  Centers  (AFOTEC)  software 
usability  evaluation. 

SUMMMIY  OF  CHANGES 

AFOTEC  Pamphlet  99-102  replaces  AFOTEC  Pamphlet  800-2,  all  volumes. 

This  volume  is  the  fourth  in  a  series  of  Software  Operational  Test  and  Evaluation  volumes  propped 
^  J  rstems  ^alysis  Directorate  at  Headquarters  (HQ)  AFOTEC.  Loc^  reproductaon  of  all 
volumes  in  this  series  is  authorized.  Comments  should  be  directed  to  the  office  of  primary 
responsibility.  The  volumes  in  the  series  are. 

AFOTEC  Pamphlet  99-102,  Volume  1  -  Management  of  Software  Operational  Test 

and  Evaluation 

AFOTEC  Pamphlet  99<102,  Volume  2  -  Software  Support  Life  Cycle  Process  Evalua¬ 
tion  Guide 

AFOTEC  Pamphlet  99-102,  Volume  3  -  Software  Maintainability  Evaluation  Guide 
AFOTEC  Pamphlet  99-102,  Volume  4  -  Software  Usability  Evaluation  Guide 
AFOTEC  Pamphlet  99-102,  Volume  5  -  Software  Support  Resources  Evaluation 

Guide 

AFOTEC  Pamphlet  99-102,  Volume  6  -  Software  Maturity  Assessment  Guide 
AFOTEC  Pamphlet  99-102,  Volume  7  -  Software  Reliability  Evaluation  Guide 

AFOTEC  Pamphlet  99-102,  Volume  8  -  Software  Operational  Assessment  Guide 

1  General  This  volume  is  divided  into  four  parts.  The  first  part  describes  the  methodology  for 
conducting  software  usability  evaluations.  The  second  part  (attachment  1)  is  an  mdex  of  items  ^d 
source  documents.  The  third  part  (attachment  2)  contains  a  complete  hstmg  of  softw^ 
evaluation  items.  The  fourth  part  (attachment  3)  consists  of  a  users  manual  for  ffie  Software 
Usability  Evaluator  (SUE)  computer  program,  an  automated  method  of  preparmg,  admmistermg, 
and  analyzing  software  usability  evaluations. 

2.  Background. 

2.1.  The  operator-software  interface  is  becoming  steadily  more  important  in  the  systems  the  Aft 
Force  develops  and  fields.  New  software  developments  (e.g.,  windows,  expert  systems,  and  object- 
orientation)  change  the  nature  of  the  interaction  between  the  human  and  computer.  So^are 
usability  evaluations  have  been  performed  by  AFOTEC  over  the  past  15  years  usmg  two  principal 
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methodologies.  Software  usability  was 
examined  from  the  suitability  perspective 
using  a  standardized  question  set  called  the 
Software  Operator-Machine  Interface  Ques¬ 
tionnaire  (SOMIQ).  The  SO^^IQ  question¬ 
naire  provided  a  method  for  determining  the 
adequacy  of  the  software  interface  based  on 
structured  interviews  conducted  with  experi¬ 
enced  software  users.  The  result  of  SOMIQ 
was  a  single  standardized  score  w’hich  could 
be  used  to  compare  the  relative  usability  of 
different  software  interfaces. 

2.2.  At  the  same  time,  software  usability  was 
also  examined  from  a  human  factors/opera¬ 
tional  effectiveness  perspective.  This  ap¬ 
proach  used  custom  operator-software  inter¬ 
face  questionnaires  developed  luuquely  for 
each  system  under  test,  and  performed  as  part 
of  the  human  factors  evaluation.  The  results 
of  these  questionnaires  were  descriptive 
statistics  relating  specific  problems  identified 
to  instances  of  mission  failure  or  degradation 
of  system  effectiveness. 

2.3.  In  recognition  of  the  overlap  in  the 
suitability  and  effectiveness  evaluations  of 
software  usability  emd  the  potential  for 
conflicting  findings,  these  two  approaches 
were  ultimately  combined  into  a  single  meth¬ 
odology  under  the  auspices  of  the  Human 
Factors  and  Training  Analysis  Team,  HQ 
AFOTEC/SAH.  The  results  of  this  consolida¬ 
tion  are  documented  in  the  present  volume. 

2.4.  There  is  a  great  deal  of  literature  on  the 
subject  of  "software  usability  engineering,"  but 
relatively  few  texts  on  the  evaluation  of  soft¬ 
ware  usability  for  developed  systems.  One  of 
the  most  frequently  cited  source  documents  is 
that  of  Smith  and  Mosier  (1986).  Smith  and 
Mosier  developed  a  comprehensive  set  of 
design  guidelines  organized  around  six  func¬ 
tional  areas.  Each  of  the  functional  areas 
represents  a  feature  or  function  of  a  software 
interface  such  as  data  entry,  data  display,  and 
user  guidance.  Similar  to  Smith  and  Mosier, 
Militaiy  Standard  (MILrSTD)-1472D  outlines 
software  interface  functions  that  fall  into  11 
categories.  These  categories  overlap  with 
those  of  Smith  and  Mosier  and  also  provide 
guidelines  for  some  additional  functions. 
MIL-STD-1801  also  contains  design  guidelines 
organized  according  to  interface  functions  and 


uses  essentially  the  same  categories  used  by 
Smith  and  Mosier. 

2.5.  A  somewhat  different  approach  was 
taken  by  another  frequently  cited  work, 
Ravden  and  Johnson  (1989).  Ravden  and 
Johnson  present  design  guidelines  organized 
around  a  set  of  nine  usability  characteristics 
or  "criteria."  These  characteristics  include 
features  such  as  visual  clarity,  consistency, 
and  compatibility.  This  approach  is  very 
similar  to  the  AFOTEC  SOMIQ  questionnaire 
described  earlier.  Both  approaches  use 
ratings  of  the  degree  to  which  individual 
instances  of  these  characteristics  or  features 
are  present  or  absent  as  their  method  of 
evaluating  the  usability  of  a  given  software 
configuration  item. 

2.6.  As  described  above,  the  literature  on 
software  usability  evaluation  can  be  roughly 
divided  into  two  approaches,  one  focusing  on 
the  functions  or  tasks  software  interfaces 
must  perform  and  the  other  organized  aroxmd 
a  number  of  usability  characteristics  that  are 
independent  of  function.  The  approaches  are 
orthogonal  in  that  they  form  two  independent 
dimensions  along  which  usability  can  be 
evaluated.  The  present  software  usability 
evaluation  methodology  described  in  this 
volume  has  been  aligned  with  both  of  these 
approaches. 

3.  Software  Usability  Evaluation  Method. 

3.1.  Methodology  Overview. 

3.1.1.  The  software  usability  evaluation  meth¬ 
odology  contains  a  database  of  177  questions 
or  evaluation  items  that  cover  the  entire 
spectrum  of  usability  characteristics  and 
interface  functions.  Each  evaluation  item  is 
cross-indexed  with  the  Ravden  and  Johnson 
and  SOMIQ  features  as  well  as  individual 
sections  of  MILrSTD-1472D,  MIL-STD-1801, 
and  Smith  and  Mosier  that  address  specific 
interface  functions.  The  process  of  creating 
and  administering  a  software  usability 
evaluation  can  be  performed  either  manually, 
with  hard  copy  forms,  or  by  computer,  using 
the  SUE  software  tool. 

3.1.2.  Performing  a  software  usability  eval¬ 
uation  for  a  specific  system  or  software  con¬ 
figuration  item  is  based  on  a  subset  of 
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questions  selected  from  the  177  evaluation 
items.  Based  on  a  description  or  profile  of  the 
software  to  be  tested,  apphcable  questions  are 
drawn  from  the  SUE  database  and  adminis¬ 
tered  to  system  operators.  Operators  rate  not 
only  whether  or  not  the  usability  elements  are 
present  or  absent  (as  with  the  SOMIQ  and 
Ravden  and  Johnson  methods),  but  also  the 
degree  of  mission  impact  of  any  usability 
problems.  This  second  rating  reflects  the 
legacy  to  the  operational  effectiveness  ap¬ 
proach  to  usability  evaluations  and  is  an 
important  link  in  establishing  the  criticality  or 
relative  importance  of  any  adverse  findings. 
In  addition  to  identifying  the  mission  impact 
of  usability  problems,  respondents  are  also 
asked  to  provide  a  brief  example  of  the 
problem. 

3.1.3.  The  evaluation  then  proceeds  by 
ranking  the  usability  problems  by  frequency  of 
occurrence  and  ratings  of  mission  impact.  For 
reporting  purposes,  the  problem  areas  and  the 
examples  can  then  be  linked  back  to  the 
applicable  design  standards  associated  with 
each  evaluation  item.  An  index  relating  each 
item  to  its  referents  in  each  of  the  above 
source  documents  is  provided  in  attachment  1. 

3.1.4.  In  summaiy,  the  current  software 
usability  evaluation  methodology  has  impor¬ 
tant  links  to  previous  AFOTEC  usability 
evaluations  and  to  external  usability  stan¬ 
dards.  Accompanied  with  a  computer  tool  to 
efficiently  generate  and  administer  the  evalu¬ 
ation  questionnaires  and  analyze  the  resulting 
data,  the  SUE,  methodology  is  a  powerful, 
state-of-the-art  technique  for  evaluating  the 
usability  of  software  interfaces. 

3.2.  Planning  the  Evaluation. 

3.2.1.  In  preparing  to  conduct  a  software 
usability  evaluation  for  a  specific  operational 
test  and  evaluation  (OT&E),  the  effort  begins 
by  selecting  an  appropriate  question  set  from 
the  database  of  evaluation  items.  This  is 
performed  either  by  the  test  team  member 
responsible  for  conducting  the  evaluation  or 
by  HQ  AFOTEC/SAH  personnel  (depending  on 
manpower  limitations  and  test  timing).  In 
either  case,  the  selection  of  a  question  set 
requires  knowledge  about  the  system  inter¬ 
face,  the  functions,  equipment,  and  operator 
or  maintainer  tasks  associated  with  the 


interface.  This  sort  of  information  is 
commonly  obtained  from  system  training 
materials,  position  handbooks,  task  and  skill 
analyses,  notes  taken  from  direct  observation 
at  site  visits,  or  through  interviews  with 
system  experts. 

3.2.2.  The  selection  process  begins  by  defining 
a  system  profile  in  terms  of  the  interface 
functions  and  associated  support  equipment. 
Completing  the  profile  definition  procedure  in 
the  SUE  software  will  identify  a  set  of 
evaluation  items  that  will  form  the  basis  of 
the  evaluation  question  set.  Note  that 
separate  item  sets  may  be  required  for  crew 
members  who  use  different  types  of  software 
applications  (e.g.,  operators  versus  main- 
tainers).  The  next  step  is  to  review  the 
evaluation  items  selected,  one  by  one,  and 
remove  any  questions  that  are  inappropriate 
to  the  system  under  test,  or  appear  redundant 
with  other  items  in  the  set.  The  number  of 
items  in  the  questionnaire  will  of  course 
depend  on  the  size  and  scope  of  the  software 
interface  being  tested.  Our  experience  is  that 
the  average  size  question  set  for  an  OT&E  has 
been  approximately  100  items. 

3.2.3.  Once  an  item  set  has  been  selected,  it 
should  be  reviewed  with  a  system  expert  to 
further  refine  the  question  set  and  to  add 
explanatoiy  notes  that  accompany  each  ques¬ 
tion.  Explanatory  notes  are  used  to  tailor 
each  question  to  the  interface  imder  test  by 
adding  unique  nomenclature  and  terminology 
used  by  the  system.  .  The  explanatory  notes 
(along  with  definitions  and  examples)  appear 
alongside  each  question  in  the  hard  copy 
format  and  at  the  bottom  of  the  screen  for  the 
on-line  administration  format.  Note  that  each 
of  the  steps  described  above  can  be  completed 
either  manually  or  by  using  the  SUE  software 
tool.  The  next  step,  however,  preparing  the 
questionnaire  for  administration,  should  be 
accomplished  using  the  SUE  software. 

3.2.4.  Preparing  the  questionnaire  for  either 
hard  copy  or  on-line  administration  is  simply 
a  matter  of  reproducing  the  appropriate 
number  of  questionnaires  to  ensure  ^at  each 
crew  member  to  be  included  in  the  evaluation 
will  have  access  to  the  questionnaire.  In  the 
case  of  hard  copy  administration,  the  ques¬ 
tionnaire  is  printed  and  reproduced  (along 
with  a  cover  sheet  and  instructions)  so  that 
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one  paper  copy  is  available  for  each  crew 
member.  In  the  case  of  on-line  administration 
multiple  computers  running  copies  of  SUE 
should  he  made-  available  to  minimize  exces¬ 
sive  queuing  or  waiting  to  complete  the  ques¬ 
tionnaire. 

3.3.  Conducting  the  Evaluation. 

3.3.1.  The  process  of  conducting  a  software 
usability  evaduation  is  much  like  conducting 
any  other  evaluation  based  on  questionnaire 
data.  The  chief  considerations  are  the  number 
of  subjects  to  include,  when  to  administer  the 
questionnaire,  and  instruction  to  the  respon¬ 
dents.  The  Operational  Test  and  Evaluation 
Questionnaire  •Handbook  (AFOTEC  Technical 
Paper  7.2)  provides  detailed  information  on 
these  and  related  subjects.  For  the  purposes 
of  the  software  usability  evaluation,  however, 
these  considerations  will  be  briefly  reviewed 
in  the  following  paragraphs. 

3.3.2.  The  first  consideration  involves  de¬ 
ciding  on  the  number  of  respondents  (person¬ 
nel  completing  the  questionnaire)  to  include  in 
the  evaluation!  The  evaluation  process  aims 
to  describe  the  significant  features  of  the 
software  interface  rather  than  produce  a 
pass/fail  grade  for  software  usability.  As  such, 
the  goal  is  to  obtain  a  representative  sample  of 
software  users  to  rate  the  interface  features  as 
they  relate  to  the  respondents’  duties.  Thus, 
the  sample  size  should  be  driven  by  the 
number  of  personnel,  required  to  accurately 
represent  a  cross-section  of  the  types  of 
personnel  who  will  use  the  operational  soft¬ 
ware,  rather  than  being  driven  on  the  basis  of 
statistical  requirements  for  reliability  or 
confidence  levels.  For  a  large  system  with 
multiple  operator  positions  or  crew  members 
simultaneously  using  the  system,  a  sample 
size  of  15  to  20  is  not  an  unreasonable  goal. 
Smaller  systems  such  as  a  piece  of  test 
equipment  may  require  as  few  as  five  to  seven 
respondents.  Once  again,  the  goal  is  to  obtain 
an  operationally  representative  sample  of 
users  across  the  spectrum  of  Air  Force 
specialty  codes  (AFSC),  experience  levels,  and 
training. 

3.3.3.  Second,  the  issue  of  when  to  administer 
the  usability  questionnaires  is  driven  by  the 
number  of  items  in  the  question  set.  Because 
of  the  length  of  time  required  to  complete  a 


100  item  questionnaire,  it  is  not  appropriate  to 
expect  crew  members  to  complete  the 
questionnaire  at  the  end  of  their  duty  shift. 
Administration  of  the  usability  questionnaire 
should  be  scheduled  to  occur  on  duty  time  and 
will  thus  require  permission  and  .official 
release  by  their  supervisors  as  a  part  of 
scheduled  test  activities.  In  order  to  avoid  any 
adverse  mission  interference,  crew  members 
may  have  to  be  scheduled  to_  take  the 
questionnaire  one  at  a  time  or  in  small  groups 
of  two  or  three.  In  some  cases,  the  question¬ 
naire  can  be  completed  across  multiple  days, 
30  to  40  questions  at  a  time,  using  either  the 
interrupt  and  resume  option  of  the  SUE 
software  or  hard  copy  forms.  The  question¬ 
naire  administration  should  be  scheduled  to 
occur  during  the  latter  part  of  the  test  efibrt, 
after  the  respondents  have  been  exposed  to 
most  of  key  aspects  of  the  software  under  test. 
A  lengthy  question  set  should  be  administered 
only  once  during  the  test;  shorter  versions  of 
the  usability  questionnaire  can  be  adminis¬ 
tered  repeatedly  to  capture  different  scenarios 
or  test  conditions. 

3.3.4.  The  third  and  final  consideration  in 
collecting  data  for  the  usability  evaluation  is 
providing  instructions  to  the  respondents.  If 
possible,  the  respondents  should  be  pre¬ 
briefed  as  to  the  method  and  purpose  for 
conducting  the  evaluation  well  in  advance  of 
actually  completing  the  questionnaire.  These 
prebriefings  can  occur  as  a  part  of  the  test 
orientation  or  overview  given  to  crews 
participating  in  the  test  effort  or  can  be 
provided  separately  as  a  part  of  a  shift  change 
or  stand-up  briefing.  The  prebrief  is  im'por- 
tant  so  that  the  participants  will  know  why 
they  are  being  asked  to  complete  the  question¬ 
naire,  when  and  where  it  will  be  administered, 
what  aspects  of  the  software  they  will  be 
asked  about,  and  how  the  results  will  be  used. 
Information  on  these  aspects,  provided  in 
addition  to  the  instructions  that  accompany 
the  actual  questionnaire,  will  increase  the 
cooperation  and  thoughtfulness  the  respon¬ 
dents  will  put  into  their  questionnaire 
answers. 

3.4.  Analyzing  the  Results. 

3.4.1.  The  analysis  effort  is  directed  toward 
identification  of  the  software  interface  fea¬ 
tures  most  fi’equently  rated  as  problems  and. 
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in  particular,  those  features  that  adversely 
affect  mission  accomplishment.  In  order  to 
efficiently  accomplish  the  above,  the  following 
analysis  path  is  recommended:  The  first  step 
is  to  rank  questions  by  the  number  of 
respondents,  identifying  them  as  a  problem. 
Once  grouped  by  number  of  respondents,  the 
individual  questions  should  be  rank  ordered 
according  to  their  impact  ratings  (by  totaling 
the  impact  score  assigned  by  each  respon¬ 
dent).  The  SUE  software  package  will 
perform  this  analysis  for  you  by  selecting  the 
"Responses  Grouped  by  Number  of  Not 
Always  Responses”  menu  option  (see  the 
users’  guide  in  Attachment  3  for  details). 

3.4.2.  The  second  step  in  the  analysis  is  to 
group  the  results  by  related  characteristics 
and  functions  of  the  software  interface.  This 
is  accomplished  by  sorting  the  ratings 
according  to  the  interface  characteristics  (e.g., 
clarity,  consistency,  and  compatibility)  and 
software  functions.  For  example,  if  the  most 
frequently  reported  and  highest  impact 
question  was  evaluation  item  number  1,  the 
characteristic  of  "descriptiveness"  and  func¬ 
tion  "data  entry"  apply.  Grouping  results  by 
characteristic  and  function  will  enable  the 
analyst  to  identify  interrelated  problems. 

3.4.3.  Once  the  results  are  grouped  so  that  a 
list  of  related  problem  areas  and  questions  has 
been  developed,  the  comments  provided  by  the 
respondents  should  be  examined  to  refine 
further  the  grouping  into  problem  areas.  This 
can  be  accomplished  by  simply  reading  the 
hard  copy  questionnaires  or  by  selecting  the 
"All  Responses"  option  on  the  SUE  software. 
The  result  of  this  step  should  be  a  list  of 
evaluation  items  and  interrelated  software 
problems  ranked  by  the  number  of  times  they 
were  selected  and  degree  of  mission  impact. 

3.4.4.  Unlike  the  SOMIQ  evaluations  of  years 
past,  an  overall  score  for  the  evaluation  is  not 
calculated.  Instead,  descriptive  statistics  are 
calculated  for  the  responses  and  the  results 
are  described  in  a  narrative  format  in 
accordance  with  AFOTEC  Instruction 
(AFOTECI)  99-101  (see  also  AFOTEC  Policy 
Update  (27  January  1994)).  Other  analysis 
techniques  being  explored  include  a  calcula¬ 
tion  of  a  normalized  index  of  the  number  of 
mission-critical  problems  identified  for  each 
system  and  the  percentage  of  respondents 


identifying  mission-critical  flaws.  At  this 
time,  these  remain  potential  analysis 
techniques  to  be  used  in  the  future;  they  are 
not  recommended  for  use  as  test  criteria. 

3.5.  Reporting. 

3.5.1.  The  principal  goal  of  the  analysis 
procedure  described  above  is  the  development 
of  a  narrative  describing  the  key  software 
usability  problems  in  terms  of  their  impact  on 
mission  efiectiveness.  The  general  format  is 
to  present  the  key  results,  ranked  by  mission 
impact  and  organized  by  system  functions  and 
characteristics.  The  amount  of  detail  to  be 
presented  in  the  report  should  be  driven  by 
the  reporting  space  available  and  the  critical¬ 
ity  of  the  software  system. 

3.5.2.  An  example  of  this  narrative  reporting 
style  is  provided  below  in  the  following  soft¬ 
ware  usability  evaluation  as  reported  for  an 
actual  GT&E  effort  (although  the  narrative 
has  been  shortened  and  the  name  of  the 
program  has  been  changed  for  purposes  of 
distributing  this  pamphlet). 

3.5.2.1.  A  total  of  eight  crew  members 
completed  detailed  Software  Usability  Elvalua- 
tions  for  the  initial  OT&E  (lOT&E)  of  the 
Advanced  Generic  Support  Equipment  (AGSE) 
system  in  accordance-  with  AFOTECPAM 
99-102,  volume  4.  The  evaluators  showed  very 
high  agreement  and  consistency  in  their 
ratings;  all  eight  crew  members  were  un¬ 
animous  in  their  independent  selections  of  the 
top  six  problem  areas.  The  results  of  the 
survey  indicated  a  few  problems  in  the 
software  usability  areas  of  software  simplicity, 
descriptiveness,  responsiveness,  and  error 
abatement.  Each  of  the  problem  areas 
identified  was  ranked  in  terms  of  their  impact 
on  mission  accomplishment.  The  most 
significant  problems  are  presented  below , 
beginning  with  the  areas  identified  as  having 
the  greatest  mission  impact. 

3.5.2.1.1.  The  attribute  rated  as  having  the 
greatest  adverse  mission  impact  was  the 
inability  to  reverse  or  imdo  commands.  Inputs 
were  executed  immediately  without  an 
opportunity  to  back  up  or  make  changes.  Any 
incorrect  entries  required  the  user  to  restart 
the  program  firom  the  entry  point  menu.  For 
example,  once  a  sequence  of  self-test  proce- 
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dures  was  entered,  there  was  no  opportunity 
to  verify  the  selection  prior  to  their  execution. 
A  second  area  of  adverse  mission  impact  was 
associated  with  unanticipated  system  halts 
and  the  lack  of  any  information  on  how  to 
recover  from  them.  According  to  the  rater's, 
system  "lockups"  were  fairly  frequent  and 
were  not  accompanied  with  any  on-screen 
information  on  system  status  or  recovery 
procedures. 

3.5.2. 1.2.  The  third,  fourth,  and  fifth  areas 
identified  were  related  to  the  manual  for  the 
AGSE.  Specifically,  the  manual  contained 
substantial  amounts  of  material  that  was 
unnecessary  and  irrelevant  to  the  completion 
of  specific  tasks.  Further,  even  routine 
operations  required  the  crew  members  to 
consult  the  manual.  For  example,  entering 
data  codes,  loading  test  program  sets  (TPS), 
and  responding  to  system  lockups,  all  required 
repeated,  frequent  consultation  of  the  manual. 
Finally,  the  lack  of  system  prompts  for  user 
inputs  required  the  users  to  consult  the 
manual  for  completion  of  nearly  every  task. 

3.5.2.1.3.  The  sixth  problem  identified  by  the 
raters  concerned  the  inability  to  interrupt 
action  sequences  and  processes  once  initiated. 
Once  begun,  processes  could  not  be  termi¬ 
nated.  For  example,  some  tests  such  as 
infrared  (IR)  seekers  have  a  time  limit  (44 
minutes)  that  may  require  early  test  termi¬ 
nation;  the  AGSE  will  not  allow  interruption 
or  early  termination.  Similarly,  the  AGSE 
delay  mode  cannot  be  interrupted  when 
changes  to  testing  are  required. 

3.5.2.2.  Finally,  for  purposes  of  submitting 
system  design  reviews/product  quality  defi¬ 
ciency  reports  (SDR/PQDR)  or  a  stand-alone 
software  usability  report,  more  detailed 
results  and  recommendations  are  sometimes 


desirable.  To  support  this  level  of  analysis, 
the  evaluation  items  corresponding  to  the 
problem  areas  identified  can  be  referenced 
back  to  the  source  documentation  on  software 
usability  guidelines  and  MIL-STDs.  Attach¬ 
ment  1  provides  an  index  of  evaluation  items 
and  design  standards  for  the  purpose  of  this 
cross-referencing. 

4.  How  to  Recommend  Changes.  If  you 
have  comments  or  suggestions  for  future 
additions  of  this  evaluation  guide,  please 
contact  Samuel  G.  Charlton,  Deputy  Chief, 
Human  Factors  and  Training  Analysis,  HQ 
AFOTEC/SAH,  8500  Gibson  Blvd  SE,  Kirtland 
AFB,  NM  87117-5558,  DSN  number  246-1375 
(commercial  (505)  846-1375),  or  by  e-mail  at 
CHARLTONSG®  HQ.AFOTEC.AF.MIL. 

5.  References: 

AFOTECP  800-2,  Volume  4,  Software 
Usability  Evaluator’s  Guide,  23  November 
1987. 

MILrSTD-1472D  Human  Engineering  Design 
Criteria  for  Military  Systems,  Equipment,  and 
Facilities,  14  March  1989. 

MIL-STD-1801  (USAF)  User !  Computer  Inter¬ 
face,  29  May  1987. 

Ravden,  S.J.  &  Johnson,  G.I.  Evaluating 
Usability  of  Human.  Computer  Interfaces:  A 
Practical  Approach.  Chichester,  England: 
EUis  Horwood,  Limited,  1989. 

Smith,  S.L.  &  Mosier,  J.N.  Guidelines  for 
Designing  User  Interface  Software.  (ESD-TR- 
86-278,  MTR  10090),.  USAF  Electronic 
Systems  Division,  1986. 


GEORGE  B.  HARRISON,  Major  General,  USAF 
Commander 
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Index  of  Evaluation  Items  and  Design  Guidelines 


Al.l.  This  attachment  contains  an  index  of  each  of  the  177  software  usability  evaluation  items.  For 
each  evaluation  item,  the  evaluation  item  number  in  the  database  is  followed  by  the  wording  of  the 
item  (the  software  attribute  to  be  rated)  and  its  qualifier  {in  brackets}.  The  item  qualifier  is  used 
when  selecting  items  based  on  a  system  profile  (see  paragraph  3.2.2). 

Ali.  For  each  item,  the  function,  the  characteristic,  and  paragraph  number  for  each  of  the  usability 
evaluation  source  documents  are  also  listed.  Functions  and  characteristics  are  used  to  group  the 
evaluation  items  and  responses  in  terms  of  similar  capabilities  or  design  features  (see  paragraphs  2.6 
and  3.4.2).  Definitions  for  each  of  the  five  usability  characteristics  (as  adapted  fiium  AFOTECP  800- 
2,  volume  4)  are  as  follows: 

Al.2.1.  Error  Abatement.  Aids  in  validating  data  and  avoiding  or  correcting  errors. 

Al.2.2.  Responsiveness.  Allows  the  user  to  effectively  direct  system  operation. 

Al.2.3.  Descriptiveness.  Provides  the  user  with  adequate  explanations  of  every  function  he/she  is 
required  to  perform  and  every  fimction  the  system  performs. 

Al.2.4.  Consistency.  The  behavior  of  the  software  corresponds  to  the  expectations  of  the  user. 

Al.2.5.  Simplicity.  Information  presented  to  the  user  is  grouped  into  short,  readily  understandable 
structures. 

A1.3.  Finally,  at  the  far  right  of  the  listing,  the  item  number  corresponding  to  the  SOMIQ  found  in 
AFOTECP  800-2,  volume  4  is  presented. 
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explanations  sufllcient  to  under¬ 
stand  the  cause  of  the  error  condition. 
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computer  processing,  the  system 
effectively  identifies  that  delay  to  the  user. 
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56  The  system  defaults  to  menu  Interactive  Slmpliclly 

that  are  most  frequently  used  or  control 
options  logically  related  to 
current  operations. 
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65  It  is  easy  to  move  among  different  Interactive  Responsiveness 
levels  of  the  menu  hierarchy.  control 
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92  Query  language  allows  the  user  Interactive  Simplicity 
to  access  desired  data  quickly  and  control 
accurately.  (QUERY  LANGUAGE} 
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124  Where  windows  are  opened  and  Data  display  Descriptiveness  2. 7. 5. 4 

closed  automatically  by  the  software, 
they  are  available  when  needed 
and  suppressed  when  not  needed. 

{WINDOWS! 


CO 

t^ 

00 

a 

ifi 

IC 

in 

in 

N 

IS 

t>| 

<N 

cs 

<N 

(N 

C 

5^ 

o 

X 

u 

Cd 

c 

3^ 

cd 

o 

S5 

> 

u 

V 

a 

o 

T5 

c 

•5 

a 

cd 

1 

•o 

Um 

O 

•w 

cd 

1 

3 

c 

i 

3 

•M 

(0 

2^0 
t  6  Q 
,  “  •2  Z 
si  S  5 

2  1  ^ 

rfi  ^ 


te  X 


^  *§  O  ^ 

l|‘|6 

■o  5  S  Q 
S  -  s  z 
>  £  c  ^ 


£  -O 

5  V 
u  £ _ 

^  I  ^ 

is  «  o 

C  ^  n 

«  ^  i 

a  S  E 


4,  « 

V  ^  S 


S5  ^  > 

t)  u 

^  ^  4; 

5  "1 

Co 

^  1 1  s 

I  S  £  .n 


CQ  u  •->« 

E  3  ? 


rT  V 

“  13  53 


CO  O 
.  'O 

"S*  s 

it  i:  v 

2^5 

I  2  I 

c^  ta)  a 
CO 


21 


MIL-STD-  MIL-S'I  D-  Ravden  &  Smllh  «i  AFOTECP 

N  Attribute  I&  fl»^ungrl _ ^ _ function  CHAf^CTERlSXlC  leoi _ I472I) _ Johnson _ Moslpr _ 800-2  Vnl4 
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152  The  user's  or  operator's  mzinual  User  guidance  Descriptiveness 
provides  complete  descriptions  of 
operating  procedures. 

IMANUAL) 


£  ^ 

E  ^ 
cn 


c  o 

V  Vi 
•O  C 

o 


Tt 


u 

2 

§ 

< 

2: 

a 


o 

6 

2 

D 

&4 


I 

13 

3 

O 


D 

QQ 

I 


ID 


cs 

N 


00 

N 


C 


(N 

O 

03 


CJ 

o 


o 

Tf 


LO 

CN 


CO 

CO 


in 

(N 

Tf 


(A 

CO 

(A 

CO 

CO 

cn 

(A 

CO 

CO 

CO 

CO 

V 

03 

.03 

03 

03 

,  13 

c 

c 

c 

c 

C 

c 

D 

c? 

03 

03 

i? 

> 

c 

03  .- 

> 

•w 

> 

> 

> 

13 

> 

CO 

Q, 

•w 

a 

03 

Qu 

2 

C 

XI 

■'  ^ 

*3 

u 

r 

0 

0 

'5. 

To 

u 

a« 

03 

03 

03 

a 

CA 

P 

c 

CO 

p 

CO 

CO 

CO 

CO 

V 

c 

0 

03 

03 

03 

03 

Q 

CO 

u 

Q 

Q 

Q 

Q 

a: 

03 

03 

03 

03 

03 

03 

03 

03 

03 

0 

0 

03 

03 

03 

03 

03 

03 

03 

c 

E 

i 

i  " 

§ 

§ 

i 

CO 

2 

CQ 

2 

2 

2 

•0 

T3 

2 

2 

2 

3 

3 

3 

3 

3 

3 

3 

3 

3 

tic 

tc 

tic 

tie 

tio 

t£ 

t^) 

XJC 

u 

w 

u 

u 

u 

u 

u 

u 

u 

03 

03 

03 

03 

03 

03 

03 

03 

03 

CO 

CO 

CO 

CO 

CO 

X 

CO 

CO 

CO 

D 

D 

D 

D 

D 

D 

D 

D 

D 

lac 

C 


T3 

ha 

2 

c 

cd 

2 

03 

CO 

4-1 

tiC 

03 

u 

1 

0 

s 

cd 

03 

i 

cd 

0 

2 

>> 

S 

CO 

Cd 

cd 

03 

fc; 

0 

X 

c 

u 

3 

4-J 

CO 

24 


162  Auditoiy  cues  (e.g.,  computer-  User  guidance  Responsiveness  3.8. 1 

generated  button  "click",  tones) 
provide  appropriate  feedback  for 
user  Inputs.  {AUDITORY} 
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Software  Usability  Evaluation  Question  Set 


This  attachment  contains  a  listing  of  the  177  software  usability  evaluation  items  along  with  the 
examples  and  definitions  associated  with  each  item.  The  listing  is  formatted  as  it  would  appear  in 
hard  copy  format  as  generated  by  the  Software  Usability  Evaluator  (SUE)  software  tool.  It  is  not, 
however,  intended  for  use  in  an  evaluation  as  is.  A  software  usability  questionnaire  used  in  an 
evaluation  must  first  be  tailored  to  the  system  to  be  tested  as  described  in  section  3.2  of  the 
pamphlet.  This  listing  is  provided  as  a  reference  guide  to  the  complete  list  of  evaluation  items,  the 
accompanying  explanatory  information,  and  the  appropriate  format  for  a  hard  copy  questionnaire. 
The  instructions  to  subjects  at  the  beginning  of  the  listing  are  representative  of  instructions  that 
could  be  used  for  any  evaluation;  the  SUE  software  does  not  provide  a  hard  copy  instruction  page. 


SOFTWARE  USABILITY  EVALUATOR 
INSTRUCTIONS  TO  RESPONDENTS 


This  questionnaire  contains  a  series  of  questions  about  the  usability  of  tiie  software 
in  the  system  you  have  been  testing.  In  general,  "usability"  simply  means  how 
easy  the  software  is  to  use.  In  this  questionnaire,  usability  will  be  assessed  in 
terms  of  the  presence  or  absence  of  a  set  of  software  quality  features  that  are 
indicative  of  good  interface  design.  For  each  assessment  item,  you  will  be  asked 
to  indicate  whether  or  not  the  listed  design  feature  is  consistently  implemented  in 
the  software  interface  being  evaluated. 

Responses  are  to  be  made  by  darkening  the  circle  adjacent  to  the  descriptor  that 
best  expresses  your  opinion.  The  initial  response  is  to  indicate  whether  the  design 
feature  is  tiihsx  Always  or  Not  Always  present  in  the  software  interface  you  are 
evaluating.  The  additional  choices  of  Don't  Know  and  Not  Applicable  may  also  be 
selected,  if  appropriate. 

Where  a  design  feature  is  lacking  (i.e.,  your  response  is  Not  Always),  please 
provide  an  estimate  of  the  difficulty  caused  by  the  absence  of  tiiat  feature.  These 
ratings  are  expressed  in  terms  of  the  impact  on  operational  effectiveness,  ranging 
from  Very  Low  to  Very  High. 

Where  a  design  feature  is  lacking,  you  are  also  asked  to  provide  an  example  of 
where  or  how  that  feature  is  lacking.  Such  an  example  might  be:  "The  cursor  is 
difficult  to  locate  on  the  data  base  screen". 

Many  of  the  assessment  items  are  accompanied  by  clarifying  information  in  the 
form  of  one  or  more  of  the  following:  (1)  A  Note  on  the  intent  of  the  question  and 
how  it  relates  to  the  system  under  evaluation;  (2)  A  Definition  of  key  terms  in  the 
question;  and  (3)  An  Example  of  successful  implementation  of  the  referenced 
design  feature.  This  information  appears  on  the  facing  pages,  aligned  with  the 
question  to  which  the  information  applies.  Use  this  information  whenever  it  can 
help  your  understanding  of  the  questions. 

Please  answer  all  question  at  your  own  pace.  If  you  have  any  uncertainty  about 
the  meaning  of  terms  or  the  intent  of  the  questions,  please  feel  free  to  consult  the 
questionnaire  administrator. 


Thank  you  for  your  assistance! 
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NOTE: 


DEFINITION: 


EXAMPLE:  A  process  requiring  specific  data  cannot  be  initiated  until  the 
appropriate  data  have  been  input. 


NOTE: 


DEFINITION: 


EXAMPLE:  There  is  a  blinking  cursor  or  other  prompt  at  the  location  on  the 
screen  into  which  data  are  required  to  be  entered. 


NOTE: 


DEFINITION: 


EXAMPLE:  The  system  gives  an  indication  as  to  the  nature  of  the  data 
(e.g./  numbers,  letters,  dates)  that  must  be  entered. 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 


1.  The  user  is  adequately  prompted  when  data  entry  is 
required . 

ALWAYS  NOT  ALWAYS  DON^T  KNOW  NOT  APPLICABLE 
0  0  O  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


2.  The  user  is  adequately  prompted  as  to  where  on  the 
display  data  are  to  be  entered. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  •  O  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


- - VEIT - ‘  VEftV 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  O  0  0 


3.  The  user  is  adequately  prompted  as  to  what  kind  of  data 
are  required  to  be  entered. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE:  If  a  social  security  number  were  required  to  be  entered,  it  would 
be  prompted  as:  nnn-nn-nnnn. 


NOTE: 


DEFINITION:  A  device  is  ’’appropriate"  if  it  minimizes  the  time  and  effort 
required  for  data  input  and  it  minimizes  the  likelihood  of 
input  error. 


EXAMPLE: 


NOTE: 


DEFINITION:  A  "cursor"  is  a  graphical  image  showing  where  keyboard-entered 
text  will  appear  or  where  a  selection  can  be  made.  HOME 
position  is  where  the  cursor  first  appears  on  a  new  screen. 


EXAMPLE:  After  a  command  is  executed,  the  cursor  always  appears  in  the 
upper  left-hand  corner  of  the  display. 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 


Where  data  entry  requires  fixed  or  maximum  allovable 
character  lengths,  there  is  appropriate  prompting  on 
entry  size. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  o  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


What  IS  the  overall  impact  of  this  LOW  LOW  MEDIUM 

feature  on  operational  effectiveness?  00  O 


VERY 
HIGH  HIGH 
0  0 


Input  devices  (e.g.,  keyboard,  cursor,  mouse,  joystick) 
are  appropriate  for  the  tasks  being  performed. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

O  O  o  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  7EE7 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


The  cursor  returns  to  a  HOME  position  that  is  consistent 
across  all  display  screens. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  ^ - Test* 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  0  0 


Page  3A 


NOTE: 


DEFINITION:  A  "cursor”  is  a  graphical  image  that  shows  where  keyboard- 
entered  text  will  appear  or  where  a  selection  can  be  made. 


EXAMPLE:  The  cursor  is  never  obscured  or  camouflaged  by  other  images  on 
the  screen.  Nor  does  the  cursor  move  "off  the  screen"  without 
some  indication  of  where  it  is  or  how  to  retrieve  it. 


NOTE: 


DEFINITION;  A  "cursor"  is  a  graphical  image  that  shows  where  keyboard- 
entered  text  will  appear  or  where  a  selection  can  be  made. 


EXAMPLE; 


NOTE: 


DEFINITION:  A  "cursor"  is  a  graphical  image  that  shows  where  keyboard- 
entered  text  will  appear  or  where  a  selection  can  be  made. 


EXAMPLE;  The  system  groups  similar  or  successive  activities  in  adjacent 
fields.  Alternatively,  the  system  may  automatically  move  the 
cursor  to  the  next  field  when  an  appropriate  input  is  made. 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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^  QUESTION/ STATEMENT 

7.  The  cursor  is  easy  to  locate. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


8.  The  cursor  can  be  quickly  and  accurately  positioned. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
.  0 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _  _ 


What  is 
feature 

the  overall  impact  of  this 
on  operational  effectiveness? 

VERY 

LOW 

0 

LOW 

0 

VERY 

MEDIUM  HIGH  HIGH 

O  0  0 

9.  Requirements  for  cursor  movement  from  one  field  to 
another  are  minimized. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  ,  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  O  0  0 


EXAMPLE : 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 
*10.  Data  fields  are  adequately  labeled. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
0 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ 


- —  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


11.  Data  entry  is  user-paced,  rather  than  system-paced. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
0 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ 


- : -  ^  ^  '  VERY  .  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  .  0  0  0  0  0 


12.  The  system  minimizes  requirements  for  keyboard  and 
keypad  data  entry. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


- Very  VEET 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE; 


NOTE; 


DEFINITION; 


EXAMPLE;  Data  inputs  can  be  reviewed  by  the  operator  before  the  system 
processes  them. 


NOTE; 


DEFINITION : 


EXAMPLE; 


The  system  allows  simple,  selective  editing,  and  minimizes  the 
requirement  to  re-enter  inputs  that  do  not  require  changes. 


SOFTWARE  USABILITY  QUESTIONNAIRE 


Page 


0 


question/ STATEMENT  _ _ 

13.  Dati'are”entered  in  faniliar  units  of  measurement. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
O 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking: _ _ _ _ _ _ — 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness. 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


14.  Data  inputs  can  be  easily  verified  before  entry  or 

ALWAYS^°"’nOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS" ,  please  provide  an  example  of  where  the 
[h  feature  is  lacking:  _ _ _ _ _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness.' 


VERY - -  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
O  0  0  O  0 


15.  Text  inputs  are  easy  to  edit. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
O  , 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness. 


LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE; 


NOTE: 


definition; 


EXAMPLE;  There  are  spaces  or  line  feeds  between  long  strings  of 
alphanumerics. 


NOTE; 


DEFINITION; 


•'Shift  or  control  key  inputs"  are  inputs  that  require  you  to 
press  two  keys  simultaneously  (e.g.,  the  CONTROL  key  along 
with  an  alphabetic  key)  to  enter  the  data. 


EXAMPLE 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 


Where  abbreviations  and  specialized  words  are  required 
for  data  entry,  these  terms  are  easily  associated  with 
normal  language  or  well-known  job  terminology. 

.ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

O  0  O.  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


What  is  the;  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


Long  alphanumeric  items  are  partitioned  for  easy  data 
entry. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


Very  7157 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


Data  entry  requires  a  minimal  amount  of  shift  or  control 
key  inputs. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE:  Data  entry  is  consistent  with  the  way  the  task  is  sequenced  or 
the  way  in  which  you  normally  think  of  the  task. 


NOTE: 


DEFINITION: 


EXAMPLE:  If  dates  are  entered  as  MM/DDD/YY  (e.g.,  Ol/JAN/93)  in  one 
instance,  then  the  same  format  is  used  wherever  a  date  is 
required  to  be  entered  by  the  operator. 


NOTE : 


DEFINITION: 


EXAMPLE:  If  the  system  were  to  display  the  rank  "SGT",  it  should  require 
you  to  enter  the  rank  as  "SGT",  rather  than  "SERGEANT". 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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19. 


QUESTION/ STATEMENT 

Data  are  entered  in  a  logical  order. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking;  _  ' _ 


-  VERY - -  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


20.  Formats  for  data  entry  are  consistent  across  different 
displays. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  O 


21.  Formats  for  data  entry  are  compatible  with  formats  for 
data  display  outputs. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ ' 


'  Very  Very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  O 


Page  8A 


NOTE: 


DEFINITION: 


EXAMPLE:  If  the  source  document  were  a  subscriber  list,  the  data  entry 
display  would  also  look  like  a  subscriber  list. 


NOTE: 


DEFINITION: 


EXAMPLE:  If  you  have  already  entered  your  date  of  birth,  you  should  not  be 
required  to  enter  your  age. 


NOTE: 


DEFINITION: 


EXAMPLE; 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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22. 


question/ STATEMENT 

Where  data  are  entered  from  source  documents,  the  format 
for  data  entry  corresponds  to  that  of  the  source 

ALWAYS^^^ *  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


23. 


The  user  is  not  required  to  enter  data  already  available 
to  the  software. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  0  o  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ 


- ^ - - - ter? - ^ 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH 

feature  on  operational  effectiveness?  00  00  0 


24 


The  system  provides  quick,  positive  feedback  on  the 
acceptance  or  rejection  of  data  entry. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  O 


For  a  response  of  "NOT  ALWAYS” ,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY VERY 
LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 
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NOTE: 


DEFINITION:  A  system  "validates"  a  user  input  when  it  performs  a  check  for 
acceptable  values  or  characters. 


EXAMPLE:  If  a  number  is  required  to  be  entered,  an  alphabetic  character 
will  be  rejected  by  the  system. 


NOTE: 


DEFINITION: 


EXAMPLE 


NOTE: 


DEFINITION : 


EXAMPLE:  The  system  asks  "ARE  YOU  SURE?"  before  executing  a  destructive 
data  entry  command. 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 

The  system  validates  user  inputs  before  processing  them. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking;  _ ^ _ 


- -  VERY  vm 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


26.  Where  the  system  requires  substantial  time  to  process  a 
data  entry,  there  is  an  adequate  "PROCESSING”  or 
"WORKING”  message. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ ^ _ 


What  is 
feature 

the  overall  impact  of  this 
on  operational  effectiveness? 

VERY 

LOW 

0 

LOW  MEDIUM 

0  0 

"VERY^ 
HIGH  HIGH 

0  0 

27.  There  are  adequate  safeguards  against  deleting  or 
overwriting  protected  data. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  O  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY 

LOW  LOW 
0  0 


MEDIUM 

O 


VERY 
HIGH  HIGH 
0  0 


Page  lOA 


NOTE: 


DEFINITION: 


A  "data  entry  error"  is  an  incorrect  alphanumeric  input. 
"Adequate  notification"  is  timely  and  easy  to  notice. 


EXAMPLE:  If  a  data  entry  error  is  made,  the  system  will  not  proceed  unless 
the  entry  is  corrected  or  deleted. 


NOTE: 


DEFINITION:  A  "data  entry  error"  is  an  incorrect  alphanumeric  input. 


EXAMPLE:  The  timing  and  content  of  data  entry  error  messages  make  it  clear 
that  it  is  a  data  entry  action,  as  opposed  to  a  system  state  or 
external  event,  that  has  triggered  the  error  message. 


NOTE: 


DEFINITION:  A  "data  entry  error"  is  an  incorrect  alphanumeric  input. 


EXAMPLE:  A  message  such  as  "TOO  MANY  CHARACTERS  FOR  FILENAME"  would  be 
preferable  to  "ERROR". 
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28. 


QUESTION/ STATEMENT 


The  system  provides  adequate  notification  when  it 
detects  a  data  entry  error. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking;  _ _ _ 


VERY  VERY 
What  is  the  overall  , impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  O  0  0 


29. 


Data  entry  error  messages  are  specific  to  the  detected 
error  condition. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ _ 


VERY  VERY 
What  is  the  overall,  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


30.  Data  entry  error  messages  provide  explanations 
sufficient  to  understand  the  cause  of  the  error 
condition. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking;  _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 

DEFINITION:  A  "data  entry  error"  is  an  incorrect  alphanumeric  input. 

pyampLE*  a  message  such  as  "FILENAME  ERROR:  MAXIMUM  FILENAME  LENGTH  IS  8 
CHARACTERS"  would  be  preferable  to  "FILENAME  ERROR".' 


NOTE : 

DEFINITION:  A  "data  entry  error"  is  an  incorrect  alphanumeric  input. 

EXAMPLE:  The  system  allows  easy  retrieval,  editing,  and  verification  of 
entries  that  need  to  be  corrected. 


NOTE: 

DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 

EXAMPLE:  The  system  displays  "PRESS  VERIFY  TO  CONFIRM"  when  it  requires 
confirmation  of  a  delete  command. 
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QUESTION/ STATEMENT 


31.  When  a  data  entry  error  is  detected,  the  system  provides 
adequate  guidance  on  how  to  correct  the  error. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


- - - VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  .  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


32.  Data  entry  errors  are  easy  to  correct. 


ALWAYS 

O 


NOT  ALWAYS 
O 


DON'T  KNOW 
0 


NOT  APPLICABLE 
O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


- -  ^  :  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  O  0  0 


33.  The  system  provides  adequate  prompts  when  sequence 
control  actions  are  required. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


- -  Very  ^  VERT 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


nFFiNITION*  '’Sequence  control”  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE:  You  are  neither  forced  to  speed  up  or  slow  down  your  commands  to 
the  system  because  of  system  response  time. 


NOTE: 


Q£P j jljip JON :  "Sequence  control''  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE;  There  is  a  visible  or  audible  response  to  each  sequence  control 
command  within  2-3  seconds,  so  you  are  not  forced  to  wonder 
whether  the  system  has  received  your  command. 


NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 


34.  Sequence  control  actions  are  user-paced  rather  than- 
svsten-paced . 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking;  _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


35.  The  system  responds  quickly  and  accurately  to  sequence 
control  actions. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
ft  feature  is  lacking:  _ _ _ _ _ = - 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


■very  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


36.  Sequence  control  requires  minimal  user  actions. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 
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NOTE: 


nFFiNiTION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 

fyampte*  If  the  command  "RUN"  were  used  to  initiate  an  action  in  one 
EXAMPLE,  then  the  sane  command  should  be  used  elsewhere,  as 

opposed  to,  for  example,  "GO"  or  "BEGIN". 


NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 

FXAMPLE*  If  a  query  of  a  subscriber  list  produced  names  in  alphabetical 

order,  all  similar  query  results  should  be  in  alphabetical  order 


NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 

Sequence  control  actions  employ  consistent  terminology. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


- - - 

What  is  the  overall  ^.mpact  of  this  LOW  LOW  MEDIUM.  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


38.  Sequence  control  actions  are  consistent  in  form  and 
results. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 


feature  is  lacking: 


-  -  ^ 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


39.  Sequence  control  can  be  performed  without  reference  to 
user's  or  operations  manuals. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


- - - ^  ^  VERY  - TEEY" 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  0  0 
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NOTE: 


DEFINITION:  ''Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE:  A  sequence  control  delay  might  be  communicated  to  the  user 

through  a  "PROCESSING"  or  "WORKING"  message.  Alternatively,  the 
delay  might  be  communicated  as  an  icon  such  as  an  hour  glass. 


NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE:  If  a  MODEM  has  a  40-second  timeout  for  initialization,  then  the 
sequence  control  actions  for  selecting  the  initialization 
parameters  should  be  less  than  40  seconds. 


NOTE: 


DEFINITION:  "Sequence  control*'  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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ION/ STATEMENT 


40.  Where  sequence  control  actions  must  be  delayed  because 
of  computer  processing,  the  system  effectively 
identifies  that  delay  to  the  user. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  .  0 


For  a  response  of  ''NOT  ALWAYS” ,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


-  ^7^ 

What  is  the.  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


41.  System  response  times  for  sequence  control  actions  are 
consistent  with  operational  requirements. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ 


— -  Vert - 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


42.  The  system  informs  the  user  when  a  requested  sequence 
control  function  is  completed,  either  successfully  or 
unsuccessfully. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


- - VERT- 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  0  0 
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NOTE : 


DEFINITION; 


"Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE : 


NOTE: 


DEFINITION; 


"Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE: 


NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE:  The  system  displays  "TERMINATE  TAPE  BACKUP,  Y  OR  N"  when  a 
command  is  issued  during  execution  of  an  earlier  command  to 
perform  tape  backup. 
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QUESTION/ STATEMENT 


Sequence  control  actions  can  be  easily  interrupted  by 
the  operator  when  necessary. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


VERY  VERY 
What  is  the  overaU  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


The  system  proyides  adequate  protection  against 
accidental  activation  of  sequence  control  functions. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  “NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


The  system  gives  adequate  warning  before  active  sequence 
control  functions  are  terminated. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ _ 


_  ; 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE:  The  system  provides  an  "ARE  YOU  SURE?"  prompt  before  the 
destructive  command  is  executed. 


NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE:  The  message  "CANNOT  COPY  A  FILE  ONTO  ITSELF"  would  be  preferable 
to  the  message  "FILE  COPY  ERROR". 


NOTE: 


DEFINITION:  "Sequence  control"  refers  to  operator  actions  that  initiate, 
terminate,  or  interrupt  a  system  process. 


EXAMPLE:  The  message  "MODEM  IN  USE,  AVAILABLE  MODEMS  ARE  2,  4,  AND  5" 
provides  the  information  necessary  to  correct  the  sequence 
control  command. 
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46. 


QUESTION/ STATEMENT 

The  system  gives  adequate  warning  before  destructive 
sequence  control  functions  are  exectued. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  0  0  o 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ ^ _ 


- - - VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  000  00 


47.  When  a  sequence  control  error  is  detected,  there  is 
adequate  explanation  as  to  the  nature  of  the  error. 
ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

o  0  o  o 


For  a  response  of  ” 
feature  is  lacking: 


NOT  ALWAYS”,  please  provide  an  example  of  where  the 


^  VERY  ^  ^  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


48.  When  a  sequence  control  error  has  been  detected,  the 
system  provides  adequate  guidance  on  how  to  remedy  the 
error  condition. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ 


- - - ^  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  O  0 
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DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE:  There  is  a  reserved  field  in  which  the  mode  (e.g.,  EDIT,  MONITOR, 
COMMAND)  is  always  displayed. 
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49. 


question/ STATEMENT 

Incorrect  and  unintended  sequence  control  actions  are 
easy  to  reverse  or  correct. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking;  _ _ ^ _ 


— - ^ ^ - -  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


50.  Where  multiple  users  interact  with  the  same  data,  there 
are  adequate  provisions  for  protecting  one  user's  data 
from  actions  by  the  other  user. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  '  O  O 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ ^ _ 


- - -  ^  ^  Very  very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  .0  0 


51.  Where  there  are  several  possible  modes  of  operation 
(e.g.,  EDIT,  MONITOR),  the  user  knows  what  mode  he  is 
currently  in. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  .0  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ 


VERY  7er7 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 
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NOTE: 


DEFINITION:  A  "menu"  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE : 


NOTE; 


DEFINITION:  A  "menu"  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE: 


NOTE: 


DEFINITION:  A  "menu"  is  a  list  of  availabie  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE:  All  file  manipulation  options  (e.g.,  create,  delete,  copy,  move) 
are  grouped  together. 
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CVESTION/ STATEMENT 

Menus  and  nenu  selection  options  are  appropriately- 
labeled. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking;  _ _ _ 


- - -  VERY  VERY 

What  is  the  overall .  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


53.  Menu  selection  options  are  readily  understandable. 


ALWAYS 

O 


NOT  ALWAYS 
O 


DON'T  KNOW 
O 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ 


^ VERY ^ VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


54.  Menu  selection  options  are  logically  organized  by 
similarity  of  function  and/or  by  order  of  use. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS" ,  please  provide  an  example  of  where  the 

fe;ature  is  lacking: 


iter?  TeRT 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 
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NOTE: 


DEFINITION:  A  ''menu”  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE : 


NOTE: 


DEFINITION:  A  ''menu”  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE:  The  defaults  for  successive  levels  of  the  menu  hierarchy  might 
be;  "FILE”,  "SAVE”,  "CURRENT  DIRECTORY". 


NOTE: 


DEFINITION;  A  "menu”  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE; 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 


55.  Menu  selection  options  are  organized  to  allow  rapid- 
access  to  frequent  conuaands. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 


feature  is  lacking: 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


56.  The  system  defaults  to  menu  options  that  are  most 
frequently  used  or  logically  related  to  current 
operations . 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

^  For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

^  feature  is  lacking:  _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VEkY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


57.  Menu  options  are  consistent  in  wording,  order,  and 
format. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  O  0 
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NOTE : 


DEFINITION:  A  ”raenu”  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE: 


NOTE : 


DEFINITION:  A  "menu”  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 

EXAMPLE: 


NOTE: 

DEFINITION;  A  "menu"  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE: 
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53. 


59. 


60. 


QUESTION/ STATEMENT 


Wording  of  menu  options  is  consistent  with  the  functions  . 
and  processes  they  control. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _  ■ _ 


- —  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  O  00 


It  is  easy  to  activate  the  desired  menu  options. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


VERY VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  O  0  0 


It  is  easy  to  locate  desired  menu  options  within  the 
menu  hierarchy. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  ^ _ _ ■ 


• - -  Very  VeKT 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


NOTE: 


DEFINITION:  A  "menu”  is  a  list  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE:  Unavailable  options  are  shown  in  gray,  while  available  options 
are  shown  in  black. 


NOTE: 


DEFINITION:  A  "menu"  is  a  list-  of  available  selections  that  a  user  may 
activate  by  keypress  or  pointing  devices. 


EXAMPLE: 


NOTE: 


DEFINITION:  The  "menu  hierarchy"  refers  to  a  tree-like  structure  of  menus 
in  which  a  given  menu  selection  leads  to  a  related  set  of  new 
menu  items,  each  of  which  may  lead  to  additional  menu  items. 


EXAMPLE:  The  system  displays  an  overlay  or  diagram  that  shows  the  user 
where  he  is  in  the  menu  hierarchy. 
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QUESTION/ STATEMENT 

Inactive  or  unavailable  menu  selections  are  either 
omitted  from  display  or  de-emphasized  to  indicate  that 
they  may  not  be  selected. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
•  O  0  O’  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


VERY 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


The  system  responds  quickly  and  accurately  to  menu 

commands .  _ 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ ^ _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  00  0 


63.  When  you  are  operating  within  the  menu  hierarchy,  it  is 
easy  to  detexnnine  where  you  are  within  that  hierarchy. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 
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NOTE: 


DEFINITION: 


The  "menu  hierarchy"  refers  to  a  tree-like  structure  of  menus 
in  which  a  given  menu  selection  leads  to  a  related  set  of  new 
menu  items,  each  of  which  may  lead  to  additional  menu  items. 


EXAMPLE: 


NOTE: 


DEFINITION: 


The  "menu  hierarchy"  refers  to  a  tree-like  structure  of  menus 
in  which  a  given  menu  selection  leads  to  a  related  set  of  new 
menu  items,  each  of  which  may  lead  to  additional  menu  items. 


FVAMPLE*  The  system  does  not  require  the  user  to  sequentia^y  back  out  of 
long  menu  hierarchies  in  order  to  access  menu  options  that  might 
reside  within  a  different  branch  of  the  hierarchy. 


NOTE: 


nFFTNiTlON:  The  "menu  hierarchy"  refers  to  a  tree-like  structure  of  menus 
in  which  a  given  menu  selection  leads  to  a  related  set  of  new 
menu  items,  each  of  which  may  lead  to  additional  menu  items. 


EXAMPLE : 
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CUESTION/ STATEMENT 


The  menu  hierarchy  is  organized  so  that  critical  or 
frequently  used  options  are  easy  to  select. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


^ VERY VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  O  0 


65. 


It  is  easy  to  move  among  different  levels  of  the  menu 
hierarchy. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


■  VERY  ^  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


66.  The  menu  hierarchy  is  organized  so  that  movement  up  and 
down  levels  of  the  hierarchy  is  minimized. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


Very  Very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


DEFINITION:  "Function  keys"  are  individual  buttons  that  permit  control  of 
the  system  by  direct  .selection  of  labeled  functions,  as 
opposed  to  selection  from  displayed  menus. 


EXAMPLE ; 


NOTE: 


DEFINITION:  "Function  keys"  are  individual  buttons  that  permit  control  of 
the  system  by  direct  selection  of  labeled  functions,  as 
opposed  to  selection  from  displayed  menus. 


EXAMPLE : 


NOTE: 


DEFINITION:  "Function  keys"  are  individual  buttons  that  permit  control  of 
the  system  by  direct  selection  of  labeled  functions,  as 
opposed  to  selection  from  displayed  menus. 


EXAMPLE:  Where  function  keys  are  defined  with  on-screen  legends,  similar 
function  keys  are  always  in  the  same  place. 
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QUESTION/ STATEMENT 

Function  keys  are  adequately  labeled. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking;  _ _ 


—  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  00  O  0  0 


68.  Function  key  locations  are  appropriate  to  their 
frequency  of  use. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking;  _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  00  0  00 


69.  The  assignment  of  functions  to  function  keys  is 
consistent  across  different  operations. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking;  _ ' 


Very  very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  000  O  0 
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NOTE: 


DEFINITION:  "Function  keys"  are  individual  buttons  that  perinit  control  of 
the  system  by  direct  selection  of  labeled  functions,  as 
opposed  to  selection  from  displayed  menus. 


EXAMPLE: 


NOTE: 


DEFINITION:  "Function  keys"  are  individual  buttons  that  permit  control  of 
the  system  by  direct  selection  of  labeled  functions,  as 
opposed  to  selection  from  displayed  menus. 


EXAMPLE:  The  system  provides  a  visible  or  audible  response  to  function  key 
inputs  within  2-3  seconds. 


NOTE: 


DEFINITION:  "Command  language"  is  a  dialogue  format  in  which  the  user 
specifies  desired  actions  through  messages  he  composes  (in 
either  natural  language  or  code)  and  transmits  to  the 


EXAMPLE:  Command  langucige  uses  English  or  English-like  expressions. 
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70. 


QUESTION/STATEMENT  _ 

The  system  responds  quickly  and  accurately  to  function 

ALWAYS^^^NOT  always  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ — 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY 

LOW  LOW 
O  0 


VERY 

MEDIUM  HIGH  HIGH 
0  0  0 


71. 


The  system  provides  adequate  feedback  when  function  key 

commands  are  initiated.  ___ 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

O  0  o  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ _ _ 


- - - - - - - -  VERY  “  VERY 

What  is  the  overall  impact  of  th^  LOW  LOW  MEDIUM 

feature  on  operational  effectiveness?  0  0  O  o  o 


72.  Command  language  is  easy,  to  remember  and  use. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY VERV 
LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


"Command  language"  is  a  dialogue  format  in  which  the  user 
specifies  desired  actions  through  messages  he  composes  (in 
either  natural  language  or  code)  and  transmits  to  the 


"Command  language"  is  a  dialogue  format  in  which  the  user 
specifies  desired  actions  through  messages  he  composes  (in 
either  natural  language  or  code)  and  transmits  to  the 


"Command  language"  is  a  dialogue  format  in  which  the  user 
specifies  desired  actions  through  messages  he  composes  (in 
either  natural  language  or  code)  and  transmits  to  the 


EXAMPLE : 
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question/statement 

73.  Corjr.and  language  entry  areas  are  consistent  across 

A^WAYS^^^  NOT^ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
"  0  O  0  O 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
•feature  on  operational  effectiveness. 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  O  0 


74 .  Wording  of  command  language  is  consistent  with  the 
functions  being  controlled. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

o  o  O  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


75.  Wording  of  command  language  is  consistent  across 
different  functions. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

o  o  o  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


LOW  MEDIUM  HIGH  HIGH 
0  0  0  0 
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NOTE: 


DEFINITION:  "Command  language"  is  a  dialogue  format  in  which  the  user 
specifies  desired  actions  through  messages  he  composes  (in 
either  natural  language  or  code)  and  transmits  to  the 


EXAMPLE:  Where  a  command  string  is  input,  and  an  error  is  recognized  by 
the  user,  the  system  permits  non-destructive  backspacing  to 
correct  the  error. 


NOTE: 


DEFINITION:  "Command  language"  is  a  dialogue  format  in  which  the  user 
specifies  desired  actions  through  messages  he  composes  (in 
either  natural  language  or  code)  and  transmits  to  the 


EXAMPLE: 


NOTE: 


DEFINITION:  "Command  language"  is  a  dialogue  format  in  which  the  user 
specifies  desired  actiohs  through  messages  he  composes  (in 
either  natural  language  or  code)  and  transmits  to  the 


EXAMPLE:  The  system  provides  a  visual  or  audible  response  to  light  pen 
inputs  within  2-3  seconds. 
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question/statement 
76.  Comrcand  language  is  easy  to  edit 


always  not  always  DON'T  KNOW  NOT  APPLICABLE 

o  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ _ — - - - 


— — - - - — - -  VERY  ^  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


77 . 


The  user  is  advised  of  command  language  errors. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS” ,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ 


- - - - - - - ^ - VERY - -  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  O 


78. 


The  system  responds  quickly  and  accurately  to  light  pen 
inputs . 

always  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

O  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  j _ : _ _ _ 


- - - ^ - - vm  . "VERY" 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


L 
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NOTE: 


offtnitioN* *  a  "touch  screen"  is  a  video  display  in  which  the  user 
^  indicates  selections  by  touching  specific  areas  on  the  screen. 


EXAMPLE: 


NOTE ; 


DEFINITION:  A  "touch  screen"  is  a  video  display  in  which  the  user 

indicates  selections  by  touching  specific  areas  on  the  screen. 


EXAMPLE:  The  system  provides  a  visual  or  audible  input  to  touch  screen 
inputs  within  2-3  seconds. 


NOTE: 


definition*  a  "track  ball"  is  a  moveable  socket-mounted  ball  used  to 

*  control  the  system  pointer.  A  "pointer"  is  a  graphical  image 
that  represents  the  current  position  of  the  track  ball. 


EXAMPLE: 
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QUESTION/ STATEMENT 

It  is  easy  to  locate  and  reach  the  appropriate  screen 
locations  for  touch  screen  inputs. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  .NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS” ,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


80.  The  system  responds  quickly  and  accurately  to  touch 
screen  inputs. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

0  0  o  o 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  O  O  0  00 


gl.  It  is  easy  to  position  the  track  ball  pointer  on  desired 
selections. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ ^ _ _ _ 
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NOTE: 


DEFINITION:  A  ’’track  ball”  is  a  moveable  socket-mounted  ball  used  to 

control  the  system  pointer.  A  ’’pointer”  is  a  graphical  image 
that  represents  the  current  position  of  the  track  ball. 


EXAMPLE:  The  system  provides  a  visual  or  audible  response  to  track  ball 
selection  button  presses  within  2-3  seconds. 


NOTE: 


DEFINITION:  A  ’’pointer”  is  a  graphical  device  that  represents  the  current 
position  of  a  mouse,  track  ball,  or  other  pointing  device. 


EXAMPLE: 


NOTE : 


DEFINITION: 


EXAMPLE:  The  system  provides  a  visual  or  audible  response  to  mouse 
selection  button  presses  within  2-3  seconds. 
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question/ STATEMENT 


82.  The  system  responds  quickly  and  accurately  to  track  ball 
selection  button  presses. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking; 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


83.  It  is  easy  to  position  the  mouse  pointer  on  desired 

ALWAYS ^°^^NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  .  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
H  feature  is  lacking:  _ _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


84.  The  system  responds  quickly  and  accurately  to  mouse 

selection  button  presses.  _ 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


LOW  MEDIUM  HIGH  HIGH 
O  O  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


•'Direct  manipulation"  is 
user  directly  acts  upon 
modifying  them,  usually 


a  dialogue  technique  in  which  the 
objects  by  selecting,  moving,  or 
by  pointing,  clicking,  or  dragging. 


EXAMPLE:  A  direct  manipulation  "object"  might  be  displayed  circuit  card, 
EXAMPLU..  selected  and  rotated,  would  reveal  the  underside  of 

the  card.  . 


NOTE: 


DEFINITION: 


fyample-  An  icon  of  a  computer  disk  might,  be  used  to  represent  a  computer 
example.  An^ic  would  be  accompanied  by  the  printed  file  name 
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9  QUESTION/ STATEMENT  _ _ _ _ 

85.  The  system  minimizes  shifting  from  mouse  to  keyboard 


inputs. 

ALWAYS 

0 


NOT  ALWAYS 
0 


DON'T  KNOW 
0 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _____ - - - — - 


What  is  the  overall  impact  of  this 

on  operational  effectiveness. 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


86.  Objects  used  in  direct  manipulation  dialogues  are  easy 

ALWAYS  ^'^‘^NOt'^ALWAYs’  DON'T  KNOW  NOT  APPLICABLE 
0  0  .  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ — - - — - 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


87  Where  the  same  object  is  used  to  represent  multiple 
entities  (e.g.,  computer  files,  directories),  the 
obiects  are  accompanied  by  distinctive  text  labels. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


■very  :  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  0  0 
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NOTE : 


DEFINITION; 


EXAMPLE : 


NOTE : 


DEFINITION; 


EXAMPLE;  If  one  object  overlaps  another,  there  is  an  easy  way  of  bringing 
the  lower  object  to  the  foreground. 


NOTE: 


DEFINITION; 


EXAMPLE:  When  data  on  a  particular  satellite  are  being  displayed,  the 
picture  of  that  satellite  goes  from  gray  to  blue. 
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QUESTION/ STATEMENT 


88.  Manipulation  of  objects  does  not  require  excessively 
fine  pointing  or  manual  adjustment. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking;  _ ^ _ 


What  is 
feature 

the  overall  impact  of  this 
on  operational  effectiveness? 

VERY 

LOW 

0 

LOW  MEDIUM  HIGH 

0  0  0 

VERY 

HIGH 

0 

89.  There  is  adequate  access  to  overlapping  objects. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
0 


NOT  APPLICABLE 
O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


- -  ; 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  O  0  0 


90.  The  status  of  manipulated  objects  (e.g.,  active, 
selected,  unavailable)  is  clearly  displayed. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  .  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


What  is 
feature 

the  overall  impact  of  this 
on  operational  effectiveness? 

VERY 

LOW 

0 

LOW  MEDIUM  HIGH 

0  0  0 

VERY 

HIGH 

0 
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NOTE: 


DEFINITION:  "Query  language"  is  a  form  of  command  language  used  to  request 
information  from  the  computer. 


EXAMPLE:  Query  language  uses  English  or  English-like  expressions. 


NOTE: 


DEFINITION:  "Query  language"  is  a  form  of  command  language  used  to  request 
information  from  the  computer. 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE 
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QUESTION/ STATEMENT 

Query  language  coitunands  are  easy  to  learn  and  remember. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ _ 


- — - VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


92.  Query  language  allows  the  user  to  access  desired  data 
quickly  and  accurately. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  .0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is 


lacking: 


- - ^  —  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  Op  0  0.0 


93.  Password  procedures  for  LOG-ON  are  easy  to  use. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  .  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 
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NOTE ; 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE:  There  is  neither  too  much  nor  too  little  information  presented  at 
one  time. 


NOTE: 


DEFINITION: 


EXAMPLE: 


When  a  "PAGE  DOWN”  command  is  issued,  the  last  row  on  the  first 
page  appears  again  as  the  first  row  on  the  second  page. 
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95. 


96. 


QUESTION/ STATEMENT 

Information  is  easy  to  find  on  the  screen. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

bo  o  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ _ _ _ 


What  is  the  overall*  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  0  0 


The  amount  of  data  presented  at  any  one  time  is 

ALWAYS*^^^^NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  examplfe  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ ^ - 


- - - - - very  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  00  0 


It  is  easy  to  follow  data  that  are  displayed  over 
multiple  pages. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking: _ _ _ _ _ 


- - -  very  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  O  00 


NOTE: 


DEFINITION 


EXAMPLE : 


NOTE: 


DEFINITION 


EXAMPLE: 


NOTE: 


DEFINITION 


EXAMPLE: 
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QUESTION/ STATEMENT 

Data  are  displayed  in  a  logical  manner. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


— - - -  veM^  very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


98.  Important  data  are  prominently  displayed. 


ALWAYS 

O 


NOT  ALWAYS 
O 


DON'T  KNOW 
O 


NOT  APPLICABLE 
O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking: 


— ^  ■  VERY  ^  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  O  O  0  0  0 


99.  In  any  given  display,  all  necessary  data  are  displayed. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


VERY  VERY 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  O  0 


1 
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NOTE : 


DEFINITION: 


EXAMPLE : 


NOTE: 


DEFINITION: 


EXAMPLE:  The  system  allows  the  user  to  limit  the  number  of  decimal  places 
displayed  in  numeric  data. 


NOTE: 


DEFINITION: 


EXAMPLE:  The  data  require  no  translation  or  manipulation  by  the  user. 
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xCO. 


QUEST I ON/ STATEMENT 

Inforraation  is  presented  to  the  user  at  a  comfortable 
rate. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking: 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0.0 


101.  Unless  requested  by  the  user,  o.nly  necessary  data  appear 
on  the  display. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  ”N0T  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


102.  Data  are  displayed  in  a  readily  usable  form. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


Very  very 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE:  Responses  to  one  user  input  are  displayed  before  the  user  has 
completed  his  next  input. 


NOTE; 


DEFINITION: 


EXAMPLE : 


Where  a  system  is  displaying  real-time  telemetry  data,  it 
"freezes”  the  values  for  10-second  intervals  to  allow  the  user 
time  to  scan  those  values. 


NOTE: 


DEFINITION: 


EXAMPLE: 
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QUESTION/ STATEMENT 


Display  update  rate  is  fast  enough  to  keep  pace  with 
user  and  system  actions. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking; 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


■very  very 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  6  0  0 


104.  Display  update  rate  is  not  so  fast  as  to  impair 
readability  or  utility  of  the  data. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  0  0  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
^mil  feature  is  lacking:  _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  O  0 


105.  Data  displays  minimize  the  requirement  for  memorization 
or  interpretation. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking;  _ _ ^ _ _ _ : - 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


■very  ^  verT 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  O  0 
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NOTE: 


DEFINITION: 


EXAMPLE:  Where  the  user  is  required  to  enter  data  from  a  paper  form,  the 
labels  and  sequence  of  items  on  the  paper  form  correspond  with 
the  data  prompts  on  the  computer  screen. 


NOTE; 


DEFINITION; 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE 
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UES  T I ON / S T AT EM ENT 


x06.  Data  display  formats  are  consistent  with  the  method  of 
data  entry. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  000  0  o 


107.  Data  displays  use  familiar  language  and  notation. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
0 


NOT  APPLICABLE 
O 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


VEM^  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  O  p  O  0  0 


108.  Data  display  formats  are  consistent  across  the  system. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  O  o 
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NOTE: 


DEFINITION; 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE : 


NOTE: 


DEFINITION: 


EXAMPLE 
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109. 


QUESTION/STATEMENT 

Data  displays  use  a  minimal  number  of  abbreviations  and 
special  characters. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  0  . 

For  a  response  of  "NOT  ALWAYS" ,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ ^ ^ _ 


- - - - -  V£gY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


110. 


Wording  is  consistent  across  displays. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS" ,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


- - - -  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


111.  Units  of  measurement  are  consistent  across  different 
displays. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 

For  a  response  of  "NOT  ALWAYS" ,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ ^ _ 


- - - -  —  yggY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE:  A  satellite  control  system  displays  all  health  and  status 
monitoring  displays  within  a  single  screen. 


NOTE: 


DEFINITION: 


EXAMPLE:  Boxes  or  lines  are  used  to  emphasize  the  separation  between 
different  data  fields. 


NOTE; 


DEFINITION: 


EXAMPLE: 
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112. 


QUESTION/ STATEMENT 

Display  grouping  is  consistent  with  system  functions. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking: _ _ _ _ _ 


- - - VERY  VERY 

What  is  the  overall  .impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  O  0  0 


113.  Data  fields  are  distinctively  separated; 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


- - - - ^ - VERY - ^  ^  VE^ 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


114.  Rows  and  columns  of  tabular  data  are  adequately 
separated . 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  .  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


VERY - -  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  0  0 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 
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NOTE: 


DEFINITION: 


EJU^PLE:  The  credit  card  number  6011256219870020  is  displayed  as: 
2562  1987  0020. 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


6011 
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QUESTION/ STATEMENT 

Long  strings  of  alphanumeric  data  are  partitioned  for 
easy  readability. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


- -  V£gY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  000  00 


116. 


Data  displays  are  organized  in  a  logical,  functionally 
useful  manner. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


- —  ^ ^ VERY ^ very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


117.  Data  displays  are  appropriately  labeled. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


- - -  VERY  ^  Ve5Y 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  O  00 
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NOTE: 


DEFINITION: 


EXAMPLE:  The  display  avoids  uncomfortable  color  contrasts  (e.g.,  yellow 
orange) . 


NOTE: 


DEFINITION: 


EXAMPLE; 


NOTE: 


DEFINITION: 


EXAMPLE 
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QUESTION/STATEMENT 

118.  Text  displays  are  easy  to  read. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking: _ _ _ 


- - — - - - -  VERY  ^  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


119.  There  is  adequate  highlighting  (e.g. ,  reverse  video, 

color,  flash,  bold  face)  of  information  requiring  user 
attention. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ 


- -  verV  ^  ^  Very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  O  0  O  0  0 


120.  Display  coding  methods  (e.g.,  reverse  video,  color, 
flash,  bold  face)  are  consistent  across  displays. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ 


- - - -  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  O  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE:  In 
be 


A  "window"  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown.  A  "window  overlay"  is  a  superimposed 
window  that  temporarily  displays  menu  choices  or  information. 


A  "window"  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


A  "window"  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


the  upper  left  corner  of  each  window  is  a  small  box  that  may 
"double  clicked"  to  close  the  window. 


121. 


122. 


123. 
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QUESTION/STATEMENT 

Windows  and  window  overlays  are  appropriate  for  the  type 
of  information  displayed  and  controlled  in  this  system. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ 


- - - - - -  VERY  ^  VERY 

What  is  the  overall  .impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


The  method  for  controlling  windows  is  consistent  across 
displays . 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


- - - -  VERY  ;  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


Windows  are  easy  to  open  and  close  manually. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  O  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ ^ _ 


- - - -  ^  Ter?" 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 
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NOTE : 


DEFINITION: 


EXAMPLE ; 


NOTE: 


DEFINITION: 


EXAMPLE : 


NOTE: 


DEFINITION: 


A  "window”  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


A  "window"  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown.  A  "window  overlay"  is  a  superimposed 
window  that  temporarily  displays  menu  choices  or  information. 


A  "window"  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


EXAMPLE: 
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QUESTION/ STATEMENT 


Where  windows  are  opened  and  closed  automatically  by  the 
software,  they  are  available  when  needed  and  suppressed 
when  not  needed. 

always  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  .  0 


For  a  response  of  “NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking; 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  O  0  0 


125.  Window  overlays  are  situated  so  that  they  do  not  obscure 
important  information. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


126.  Windows  are  adequately  labeled  as  to  the  nature  of  the 
data  presented  or  the  type  of  operation  to  be  performed 
in  that  window. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking;  _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE : 


DEFINITION:  A  "window”  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


EXAMPLE:  The  border  around  a  window  is  highlighted  when  that  window 
becomes  active. 


NOTE: 


DEFINITION:  A  "window"  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


EXAMPLE: 


NOTE: 


DEFINITION:  A  "window"  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


EXAMPLE:  A  scroll  bar  is  used  to  move  up  or  down  within  a  window. 
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QUESTION/ S 


TATEMEiNT 


127.  Where  several  windows  are  displayed  simultaneously,  it 
is  easy  to  see  which  windows  are  active  and  inactive  at 
any  given  time. 

ALWAYS  NOT  ALWAYS  DON'T  KJiOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


- -  VERY  VERT 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


128.  Where  several  windows  are  displayed  simultaneously,  it 

is  easy  for  the  user  to  shift  among  them  to  select  which 
window  is  to  be  made  active. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of 


"NOT  ALWAYS", 


please  provide  an  example  of  where  the 


feature  is  lacking: 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  O  0  0 


129.  It  is  easy  to  scroll  within  windows. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking: 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VEm - - - TEST 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 
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NOTE: 


DEFINITION:  A  "window**  is  a  display  frame  within  which  a  set  of  data  or 
menu  options  are  shown. 


EXAMPLE:  A  word  processor  provides  a  scroll  bar  "elevator"  whose  position 
within  the  scroll  bar  corresponds  to  the  position  of  the  current 
line  relative  to  the  entire  document. 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 


Page  44 


^1^  question/ STATEMENT 


When  scrolling  across  or  through  windows,  the  system 
provides  adequate  indication  of  the  position  of  the 
current  view  in  relation  to  the  area  as  a  whole. 
ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  0  0  o 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 


feature  is  lacking: 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


131.  Schematic  and  pictorial  displays  are  clearly  drawn  and 

ALWAYS*^*  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  O 

^  For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ _ 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


■vm  VEIT 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  O  0 


132.  Multiple,  simultaneous  display  graphics  do  not  interfer 

with  one  another.  _ 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM 

feature  on  operational  effectiveness?  0  0  0  o  o 
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NOTE: 


DEFINITION: 


EXAMPLE:  Operational  conventions  night  include  circle  for  friendly  track 
and  cross  for  hostile. 


NOTE: 


DEFINITION; 


EXAMPLE:  An  image  of  a  file  folder  is  used  to  represent  a  sub-directory 
a  computer  disk. 


NOTE; 


DEFINITION: 


EXAMPLE 
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QUESTION/ STATEMENT 

Display  shapes  and  symbols  conform  to  accepted  . 
operational  conventions. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _  ' _ 


VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  O  o 


134. 


Graphic  symbology  is  appropriate  for  the  information  it 
represents. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  ”NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  .0  0  0  0 


135.  Icons  and  graphic  symbols  are  used  consistently  across 
different  displays. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  O  .  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


—  -  yjRY  VeET 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE : 


NOTE; 


DEFINITION: 


EXAMPLE: 


NOTE; 


DEFINITION; 


EXAMPLE;  Operational  conventions  might  include  red  for  "danger”  and  yellow 
for  "caution". 
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136. 


question/ STATEMENT 

All  graph  axes  and  scales  are  adequately  labeled. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ 


- - - - - - - — very  -  vOT 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


137. 


Colors  used  in  displays  are  easy  to  distinguish  from  one 
another. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature 


is  lacking: 


- - - VERY  VE^ 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  O  0 


138.  Display  colors  are  consistent  with  operational 
conventions . 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

O  0  0  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ 


- — - -  VERY  ~~  VErT 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


Page  47A 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE : 


NOTE: 


DEFINITION:  "Peripheral  devices"  are  devices  connected  to,  but  not  an 

integral  part  of  the  computer  system  (e.g.,  printer,  modem, 
fax) . 


EXAMPLE:  The  system  informs  the  user  when  a  printer  is  not  on  line. 
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139. 


QUESTION/ STATEMENT 

Color  coding  is  used  consistently  across  different  • 
displays. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ ; _ 


• - -  VERY  ■■  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  O  00 


140.  The  use  of  color  enhances  the  readability  and  utility  of 
displays. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _  ■ _ 


What  is 
feature 

the  overall  impact  of  this 
on  operational  effectiveness? 

VERY 

LOW 

0 

LOW  MEDIUM  HIGH 

0  0  0 

VERY 

HIGH 

0 

141.  The  system  provides  an  adequate  amount  of  feedback  on 
the  status  of  peripheral  devices. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  0  .  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


- - -  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


NOTE 


DEFINITION 


EXAMPLE: 


NOTE: 


DEFINITION 


EXAMPLE : 


NOTE: 


DEFINITION 


EXAMPLE 
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question/ STATEMENT 

The  system  provides  adequate  feedback  when  an  internal 
fault  is  detected. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  o  o  o 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ _ 


- - - - -  VERY  ^  :  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


The  causes  of  system  halts  are  adequately  explained. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  O  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ _ _ 


- - - VERY  ^  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  O  O  O  O  0 


System  messages  to  the  user  are  informative  and  concise. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ '  _ _ _ 


— - - ^ ^ - - —  very  '  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  O  0  0  0  0 


Page  49A 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION:  “On-line  user  guidance”  consists  of  explanatory  information 

(e.g.,  indexes,  definitions,  "HELP”  functions)  provided  to  the 
user  by  the  computer  while  an  application  program  is  running. 


EXAMPLE:  The  system  provides  user-selectable  levels  of  HELP  or 
definitions. 


NOTE: 


DEFINITION:  "On-line  user  guidance”  consists  of  explanatory  information 

(e.g.,  indexes,  definitions,  "HELP”  functions)  provided  to  the 
user  by  the  computer  while  an  application  program  is  running. 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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145. 


QUESTION/ STATEMENT 


Throughout  all  phases  of  operation,  there  is  adequate 
feedback  on  system  status. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


146. 


The  system  provides  an  adequate  amount  of  on-line  user 
guidance. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _  _ _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  000  0  0 


147.  On-line  user  guidance  provides  useful  information. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking: 


Very  very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  O  0 
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NOTE : 


DEFINITION:  "On-line  user  guidance"  consists  of  explanatory  information 

(e.g.,  indexes,  definitions,  "HELP"  functions)  provided  to  the 
user  by  the  computer  while  an  application  program  is  running. 


EXAMPLE:  On-line  user  auidance  is  "standalone",  in  that  it  requires  no 
additional  written  support  to  be  of  use. 


NOTE: 


DEFINITION:  "On-line  user  guidance"  consists  of  explanatory  information 

(e.g.,  indexes,  definitions,  "HELP"  functions)  provided  to  the 
user  by  the  computer  while  an  application  program  is  running. 


EXAMPLE:  Warnings  and  caution  messages  within  on-line  user  guidance  are 
printed  in  bold  face. 


NOTE: 


DEFINITION:  "On-line  user  guidance"  consists  of  explanatory  information 

(e.g.,  indexes,  definitions,  "HELP"  functions)  provided  to  the 
user  by  the  computer  while  an  application  program  is  running. 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION  /  STATEMENT 

On-line  user  guidance  is  readily  understandable. 


ALWAYS  NOT  ALWAYS  DON'T  KIJOW  NOT  APPLICABLE 
0  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


^  VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIITM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


149. 


Critical  on-line  user  guidance  is  adequately 
highlighted. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  .0  0 

For  a  response  of  ”N0T  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  .0  O  0  0 


150.  Formats  for  on-line  user  guidance  are  consistent  across 
different  functions. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


DEFINITION:  ”On-line  user  guidance”  consists  of  explanatory  information 

(e.g.,  indexes,  definitions,  "HELP"  functions)  proyided  to  the 
user  by  the  computer  while  an  application  program  is  running. 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE 
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QUESTION/ STATEMENT 


151.  Wording  is  consistent  across  different  types  of  on-line 
user  guidance  messages. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ 


- - —  —  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW,  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


152.  The  user's  or  operator's  manual  provides  complete 
descriptions  of  operating  procedures. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ 


- -  VERY  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


153.  The  user's  or  operator's  manual  provides  only  necessary 
information. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ ' 


- -  '  VeRV 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  O  0 


NOTE: 


DEFINITION 


EXAMPLE: 


NOTE: 


DEFINITION 


EXAMPLE: 


NOTE: 


DEFINITION 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 

The  user's  or  operator's  manual  is  easy  to  use. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  '*N0T  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


_  ■ 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


155.  Guidance  in  the  user's  or  operator's  manual  is 
consistent  with  how  the  system  actually  works. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


__  ^  __  ^  __ 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  O  00 


156.  Advisory  messages  are  relevant  to  the  tasks  being 
performed. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


VERY  VEE7 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE 
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QUESTION/ STATEMENT 

Advisory  messages  are  short  and  to  the  point. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


VEM^  ve^ 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


Alarms  are  distinctive  and  easy  to  notice. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT,  APPLICABLE 

o  o  o  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ _ 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  0  0  0  0  0 


159.  The  meaning  of  alarms  is  clear. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  O  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE:  Once  acknowledged,  an  auditory  alarm  will  remain  silent  until  the 
next  alarm  condition  occurs. 


NOTE; 


DEFINITION: 


EXAMPLE 
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160. 


QUESTION/ STATEMENT 

The  user  knows  what  to  do  when  an  alarm  is  presented. 


ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


- - - —  VERY  !  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


161.  It  is  easy  to  acknowlege  and  turn  off  an  alarm  after  its 
alerting  purpose  has  been  served. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  O  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


What  is 
feature 

the  overall  impact  of  this 
on  operational  effectiveness? 

VERY 

LOW 

0 

LOW 

0 

VERY 

MEDIUM  HIGH  HIGH 

0  0  0 

162.  Auditory  cues  (e.g.,  computer-generated  button  "click", 
tones)  provide  appropriate  feedback  for  user  inputs. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  ; _ ^ _ 


I 


- - - - - gggy 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


NOTE: 


DEFINITION 


EXAMPLE: 


NOTE: 


DEFINITION 


EXAMPLE: 


NOTE: 


DEFINITION 


EXAMPLE: 
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163  . 


QU EST I ON / S T AT EMENT 


Auditory  alarms  are  effective  in  directing  the  user's 
attention  to  critical  conditions,  but  do  not  interfere 
with  other  operations. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  o 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  00  0  00 


164.  System  functions  are  organized  in  a  manner  that  is 

consistent  with  the  tasks  they  are  designed  to  perform. 
ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


—  ^  ^  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  00  0  00 


165.  System  functions  are  organized  in  a  manner  that  is 
compatible  with  how  the  user  thinks  of  his  tasks. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  O  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ 


What  is  the  overall  impact  of  this 
feature  onperational  effectiveness? 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  O  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE : 


NOTE: 


DEFINITION: 


EXAMPLE : 


NOTE: 


DEFINITION:  "On-line  training"  is  instructional  programming  that  is 

presented  through  the  operational  hardware  (as  opposed  to 
instruction  provided  through  a  training  simulator) . 


EXAMPLE: 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 


166.  The  system  provides  the  user  with  all  of  the  information 
needed  to  perform  required  tasks. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 
feature  is  lacking:  _ ; _ 


^  VERY  vm 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  00  O  0  0 


167.  The  user  is  provided  adequate  status  information  on 
transmitted  messages. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking: 


VERY  VERY 
What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 
feature  on  operational  effectiveness?  00  0  0  0 


168.  On-line  training  and  training  data  are  readily 
distinguishable  from  operational  information. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  0  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ _ _ 


■  ^  Very  very 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  000  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE 


SOFTWARE  USABILITY  QUESTIONNAIRE 
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QUESTION/ STATEMENT 


The  system  provides  adequate  protective  measures  to 
ensure  data  security. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
o  0  0  0 

For  a  response  of  "NOT  ALWAYS”,  please  provide  an  example  of  where  the 


feature  is  lacking: 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


170.  The  system  provides  adequate  warning  when  data  security 
is  threatened. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 

o  0  .0  O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


171.  Security  safeguards  and  procedures  do  not  impede 
efficient  and  effective  system  operation. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  O 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ ^ _ _ _ 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE 
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QUESTION/ STATEMENT 

172.  The**system  can  be  operated  without  resorting  to  manual 


calculations  or  processing. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW 


NOT  APPLICABLE 
0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking; 


What  is  the  overall  impact  of  this 

operational  effectiveness. 


VERY  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


173.  The  system  can  be  operated  without  special  workarounds. 


ALWAYS 

O 


NOT  ALWAYS 
0 


DON'T  KNOW 
O 


NOT  APPLICABLE 
O 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 


feature  is  lacking; 


What  is  the  overall  impact  of  this 
feature  on  operational  effectiveness? 


ITERY - -  VERY 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 


174.  Routine  operations  can  be  performed  without  reference  to 
user's  or  operator's  manuals. 

Sways  not  always  don't  know  not  applicable 

o  0  o  .  o 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking;  _ _ _ _ _ _ — - - 


What  is  the  overall  impact  of ^ this 

on  opcrationsil  of fcctivGnBSS? 


■WRi - -  vert 

LOW  LOW  MEDIUM  HIGH  HIGH 
0  0  0  0  0 
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NOTE: 


DEFINITION: 


EXAMPLE: 


NOTE: 


DEFINITION: 


EXAMPLE; 


NOTE: 


DEFINITION: 


EXAMPLE 
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QUESTION/STATENENT 


175.  The  system  allows  operator  functions  to  be  performed  in 
an  optimal  manner. 

ALWAYS  NOT  ALWAYS  DON'T  KIJOW  NOT  APPLICABLE 
0  0  0  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ _ 


- -  VERY 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


176.  Operator  workload  is  appropriate  (i.e.,  neither  too  much 
to  handle,  not  too  little  to  maintain  concentration). 
ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
O  0  O  0 


For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 
feature  is  lacking:  _ 


very" 

What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  0  0  0  0  0 


177.  Functions  that  the  user  has  not  performed  for  a  period 
of  time  are  easily  re-learned. 

ALWAYS  NOT  ALWAYS  DON'T  KNOW  NOT  APPLICABLE 
0  0  O  0 

For  a  response  of  "NOT  ALWAYS",  please  provide  an  example  of  where  the 

feature  is  lacking:  _ 


What  is  the  overall  impact  of  this  LOW  LOW  MEDIUM  HIGH  HIGH 

feature  on  operational  effectiveness?  O  O  O  0  0 


(This  page  intentionally  left  blank.) 
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AFOTECPAM  99102  Volume  4  Attiichment  3  15  June  1994 


SUE  Software  Tool  Users'  Manual 


This  attachment  contains  the  users'  guide  for  installation  and  operation  of  the  Software  Usability 
Evaluator  (SUE)  software  tool  Version  2.6.  This  users'  guide  can  be  used  as  a  stand-alone  document 
for  operating  the  SUE,  but  it  does  not  provide  any  information  regarding  the  procedures  used  by 
AFOTEC  to  plan,  conduct,  analyze,  and  report  a  software  usability  evaluation  for  OT&E.  If  this  copy 
of  AFOTECPAM  99-102,  volume’ 4  did  not  contain  diskettes  with  the  SUE,  please  contact  your  local 
software  usability  point  of  contact,  or  call  or  write  the  Human  Factors  and  Training  Analysis  Team, 
HQ  AFOTEC/SAH,  8500  Gibson  Blvd  SE,  Kirtland  AFB,  New  Mexico,  87117-5558,  DSN  246-1375  or 
(505)  846-1375. 
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SOFTWARE  USABILITY  EVALUATOR.  VERSION  2.6 


USER'S  GUIDE 


SECTION  1 
INTRODUCTION 


1.1  BACKGROUND 

The  Software  Usability  Evaluator  (SUE)  is  a  microprocessor-based  tool  for  the  generation, 
administration,  analysis,  and  reporting  of  questionnaires  on  software  usability.  SUE  was 
developed  by  the  Air  Force  Operational  Test  and .  Evaluation ,  Center  (AFOTEC)  for  the 
assessment  of  usability  in  an  operational  test  and  evaluation  environment.  SUE  is  therefore 
intended  to  be  used  on  relatively  mature  software  interfaces,  where  trained  and  experienced 
users  may  be  asked  to  evaluate  both  the  interface  and  its  effectiveness  in  an  overall  mission 
context. 

SUE  is  designed  for  both  office  and  field  applications.  While  it  is  advisable  to  have  a  test 
administrator  present  to  initialize  the  software  and  manage  the  data,  SUE  may  be  self- 
administered  with  no  external  guidance  when  necessary.  Although  SUE  may  be  applied  as  a 
stand-alone  evaluation  tool,  it  is  highly  recommended  that  SUE  results  be  accompanied  by 
performance  data  (e.g„  task  speed  or  accuracy)  that  provide  objective  measures  of  software 
effectiveness. 

1.2  EVALUATION  METHODOLOGY 

As  an  operational  test  tool,  SUE  is  intended  to  provide  descriptive,  as  opposed  to 
normative,  data  on  software  usability.  The  evaluation  methodology  employs  a  set  of  software 
usability  attribute  statements  which  characterize  good  software-operator  interfaces  (e.g.,  "The 
cursor  is  easy  to  locate.").  Each  respondent  is  asked  to  indicate  whether  the  attribute  is  either 
universally  present  (i.e.,  a  response  of  "Always")  or  not  universally  present  (i.e.,  a  response  of 
"Not  Always")  in  the  interface  being  evaluated.  Where  the  attribute  is  judged  to  be  Not  Always 
present,  the  respondent  is  further  asked  to  provide:  (1)  an  example  or  instance  in  which  the 
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attribute  is  lacking;  and  (2)  a  rating,  on  a  5-point  scale,  of  the  operational  impact  of  this 
interface  deficiency. 

Since  the  SUE  is  designed  to  be  applicable  to  a  wide  variety  of  systems  and  software 
interfaces,  the  set  of  usability  attributes  is  quite  large  and  varied.  One  of  the  key  features  of 
SUE  is  the  ability  it  provides  the  analyst  or  administrator  to  tailor  the  attribute  set  for  a  specific 
system.  Tailoring  is  generally  a  two-step  process.  The  first  step  is  an  automated  query  of  the 
attribute  list  based  on  the  presence  or  absence  of  24  system  features  comprising  what  is 
referred  to  as  a  ‘System  Profile."  An  example  of  such  a  feature  is  “Light  Pen."  If,  for  example, 
the  system  under  evaluation  did  not  possess  a  light  pen,  all  usability  attribute  statements 
referencing  a  light  pen  would  be  eliminated  from  the  questionnaire  tailored  for  that  system. 


The  second,  and  optional,  step  of  questionnaire  tailoring  is  an  item-by-item  review  of  the 
list  of  attribute  statements  remaining  after  initial  tailoring  by  system  feature.  At  this  point,  the 
analyst  or  administrator  further  culls  from  the  list  of  attribute  statements  those  items  that  do  not 
pertain  to  the  system  under  consideration  and  those  items  that  are  of  lesser  interest  to  him. 

Each  attribute  statement  may  be  accompanied  by  amplifying  information,  in  the  form  of  a 
Note,  Definition,  or  Example,  that  is  intended  the  clarify  the  meaning  of  the  attribute  statement. 
Definitions  and  Examples  are  self-contained  in  SUE  files,  but  Notes  are  generated  by  the 
questionnaire  analyst  or  administrator  after  the  attribute  list  has  been  appropriately  tailored. 

SUE  may  be  administered  on-line  to  a  respondent  interacting  directly  with  a 
microcomputer^  In  this  case,  all  responses,  examples,  and  impact  ratings  are  input  directly  by 
the  respondent.  Where  a  microcomputer  is  unavailable  for  administration,  SUE  can  generate  a 
paper  analog  of  the  on-line  questionnaire.  Once  completed,  the  analyst  may  score  the  results 
manually,  or  he  may  use  SUE  routines  to  input  the  data  into  a  microcomputer  for  automated 
analysis  and  report  generation. 

A  typical  progression  of  SUE  use  proceeds  as  follows: 

1)  The  analyst  identifies  and  describes  the  system  under  study. 

2)  The  analyst  defines  a  system  profile. 
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3)  A  set  of  attribute  statements  (or  'questions*)  is  generated  by  SUE  based  on  the 
System  Profile. 

4)  The  analyst  reviews  and  edits  out  additional  questions  that  are  irrelevant  or  not  of 
interest  to  him. 

5)  The  analyst  creates  customized  notes  to  accompany  the  questions  he  has 
selected  for  the  questionnaire. 

6)  The  questionnaire  is  administered  on-line  to  one  or  more  respondents. 

7)  The  analyst  selects  response  files  for  analysis. 

8)  The  analyst  selects  and  prints  a  report  format. 

1.3  SYSTEM  REQUIREMENTS 

The  system  requirements  for  running  SUE  are  listed  below: 

1)  100%  IBM-compatible,  protected  mode  capable,  386  or  above  PC  (either  Desktop 
or  Laptop/Notebook). 

2)  A  hard  disk  and  floppy  disk  drive  (3  1/2'  or  5  1/4'). 

3)  Minimum  of  2  MB  RAM  required  for  Paradox  4.0  Runtime  (4  MB  preferred). 

4)  Minimum  of  2.5  MB  free  disk  space. 

5)  DOS  3.0  or  higher. 

6)  Mouse  control  is  supported  but  not  required;  Keyboard  entry  is  required. 

7) '  Monochrome  or  color  monitor  -  compatible  MDA,  CMGA,  CGA,  EGA,  VGA  or 
SVGA  monitor  with  adapter. 
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8)  Compatible  with  most  widely  used  printers. 


1.4  INSTALLATION  AND  STARTUP 

1 .4.1  Installing  Paradox  4.0  Runtime 

NOTE;  If  PARADOX  4.0  RUNTIME  is  already  installed  on  your  computer,  skip  this  step, 
and  go  bn  to  installing  the  SUE  software. 

1)  Insert  the  PARADOX  4.0  RUNTIME  3.5-inch  diskette  into  your  computer’s  disk 

drive. 

2)  Make  your  hard  disk  drive  the  current  drive  (if  your  hard  disk  is  drive  C,  make  sure 
the  system  prorript  is  C:\>). 

3)  Type  a;install  (or  b;instail  for  drive  B)  and  press  Enter. 

4)  Follow  the  onscreen  instructions.  If  you  have  trouble  reading  the  text  on  the 
screen,  press  ESC  to  exit  the  program.  Restart  the  program  with  the  command  arinstall  -b&w 

5)  If  the  installation  program  changed  your  Config.sys  file,  remove  the  PARADOX 
diskette  and  reboot  your  computer  before  proceeding. 

1.4.2  Installing  the  SUE  Application  Software 

1)  Insert  the  SUE  Application  diskette  into  drive  A  (or  B). . 

2)  At  the  C:\>  prompt,  type,  the  following: 

C:\>  copy  a:8ue.bat  c: 

3)  Change  directories  to  the  PARADOX  4.0  RUNTIME  directory  with  the  following 
command  from  the  DOS  prompt; 

C:\  >  cd  pdoxrun 
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4)  Copy  the  files  into  the  PARADOX  RUNTIME  directory  with  the  following 
command: 


C:\PDOXRUN  >  xcopy  a:*.* 

1 .4.3  Starting  the  SUE  Application 

To  start  the  SUE  application,  type  the  following: 

C:\  >  sue 

I 

[You  can  also  start  the  application  manually  by  changing  directories  to  the  PARADOX 
RUNTIME  directory  and  typing  the  following:  C:\PDOXRUN  >  PDOXRUN  MAIN  ] 

Once  you  have  entered  this  command,  the  Main  Menu  will  appear. 

1.4.4  Running  the  Application  from  an  MS-Windows  Environment 

Although  it  Is  not  recommended,  you  can  run  the  SUE  application  software  from  within 
Windows.  If  you  are  running  the  SUE  applications  from  MS-Windows,  you  first  select  the  MS- 
DOS  icon  to  exit  the  windows  shell.  Then,  follow  the  steps  below: 

«winshell»  C:\windows  >  CD\ 

«winshell»  C:\  >  CD  \PDOXRUN 

«winshell»  C:\PDOXRUN  >  PDOXRUN  MAIN 

1.5  FUNCTIONAL  ORGANIZATION 

SUE  functions  are  organized  in  a  hierarchical  menu  structure.  A  Main  Menu  appears  at 
the  top  level  of  the  hierarchy.  Five  paths  emanate  from  the  Main  Menu,  which  provide 
capabilities  for  questionnaire  construction,  hardcopy  administration,  online  administration, 
analysis/report  generation,  and  performance  of  various  SUE  utilities.  The  menu  hierarchy  is 
listed  below: 
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Main  Menu 

Questionnaire  Construction 
Create  New  Profile 
Edit  Questionnaire 
Print  Question  List 
Print  System  Profile 
Edit  Questionnaire  or  Profile 
Question  Set 

Print  the  Question  List 
System  Profile 
Delete  Profile 
Copy  Profile 
Print  Profile  • 

Print  Question  List 
Print  Questionnaire  Form 
Online  Administration 
Read  Instructions 
Begin  the  Questionnaire 
Resume  the  Questionnaire 
Analysis  &  Reports 
Use  Full  Data  Set 
Custom  Notes 
Use  Subset  of  Data 
Utilities 

Enter  Data  from  Hard  Copy 
Data  Management 

Verify  Respondent  Data 
Create  a  Full  Data  Set 
Create  a  Specific  Data  Subset 
Delete  a  Respondent  File 

The  balance  of  this  manual  is  organized  by  the  main  functional  levels  within  SUE, 
corresponding  to,  the  first-level  selections  within  the  menu  hierarchy. 
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SECTION  2 
MAIN  MENU 


The  Main  Menu  appears  upon  startup  of  the  SUE  software.  The  screen  is  illustrated  below: 


SOFTWARE  USABILITY  EVALUATOR 
VERSION  2.6 

— '  "■'■I- . .  .  — g-g-E-— M—— gegg-ga...  


MAIN  MENU 

Questionnaire  Construction 

> 

Print  Questionnaire  Form 

On-line  Administration 

Analysis  &  Reporting 

Utilities 

> 

Exit 

Use  arrow  keys  to  highlight  your  choice,  then  press  ENTER; 
or  use  the  mouse  to  click  on  your  choice. 


Create  or  edit  Profile  of  System  Features  to  tailor  SUE 


Highlighted  selections  (shown  here  as  bold  type)  indicate  the  selected  option  (i.e.,  the  option 
that  will  be  activated  when  "Enter"  is  pressed).  The  Information  at  the  bottom  of  the  screen 
provides  additional  detail  on  the  functions  of  the  highlighted  selection.  Selections  are  made 
either  by  pointing  and  clicking  on.  the  desired  option  with  a  mouse  or  by  highlighting  the 
selection  using  the  down  and  up  cursor  keys  and  then  pressing  "Enter." 
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SECTION  3 

QUESTIONNAIRE  CONSTRUCTION 

The  Questionnaire  Construction  option  is  used  for  the  creation  and  modification  of 
questionnaires.  This  is  normally  performed  by  a  combination  of  selection  by  system  feature 
and  item-by-item  deletion  of  unwanted  questions  (see  Section  1.2).  Selection  of  the 
Questionnaire  Construction  option  produces  the  cascaded  menu  of  options  shown  below: 


SOFTWARE  USABILITY  EVALUATOR 
VERSION  2.6 


MAIN  MENU 

Questionnaire  Construction 
Create  New  Profile 

Edit  Questionnaire  or  Profile 

Delete  Profile 

Copy  Profile 

Print  Profile 

Print  Question  List 

Cancel  _ 


Use  arrow  keys  to  highlight  your  choice,  then  press  ENTER; 
or  use  the  mouse  to  click  on  your  choice. 


Create  a  New  System  Profile 


The  function  of  each  of  these  options  is  as  follows: 

Create  New  Profile:  Create  a  new  System  Profile. 

Edit  Questionnaire  or  Profile:  Make  changes  to  an  existing  system  profile  or  question  set. 
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Delete  Profile;  Erase  an  existing  system  profile. 

Copy  Profile:  Copy  and  rename  an  existing  system  profile. 
Print  Profile;  Print  a  listing  of  the  system  profile. 


Print  Question  List:  Print  a  condensed  listing  of  questions  making  up  a  questionnaire. 


Cancel:  Return  to  the  Main  Menu. 
3.1  CREATING  A  NEW  PR'OFILE 


A  new  profile  must  be  created  any  time  a  new  questionnaire  is  to  be  written.  Creation  of  a 
new  profile  requires  naming  and  description  of  the  new  questionnaire,  followed  by  an 
enumeration  of  hardware  and  interface  features  possessed  by  the  system  being  evaluated. 
Selection  of  the  Create  New  Profile  option  produces  the  screen  shown  below: 


CREATE  NEW  SYSTEM  PROFILE 


SYSTEM  INFORMATION: 


Profile  Name: 
System 

Description:  j 
Author:  ■ 


Type  in  System  information 


Press  F2  to  Save  and  Continue 
Press  ESC  to  Cancel _ 
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The  Profile  Name  must  be  a  unique  7-character  string,  with  no  spaces  or  punctuation.  This 
is  the  name  by  which  a  particular  profile  and  subsequent  questionnaire  will  be  identified  in  all 
other  transactions.  The  Description  field  is  optional  and  may  be  used  to  provide  additional 
information  on  the  purpose  or  target  system  for  the  questionnaire.  The  Author  field  is  also 
optional,  but  it  is  advised  that  questionnaire  authors  provide  this  information  for  archival 
purposes.  Pressing  F2  saves  the  system  information  and  leads  the  user  to  the  feature  list  by 
which  the  user  characterizes  the  system  for  questionnaire  tailoring.  The  feature,  list  screen  is 
shown  on  the  following  page. 

NEW  SYSTEM  PROFILE  NAME:  ALPHA1 2 
FEATURES: 


>  Alarms 

> 

Keyboard 

> 

Peripheral  Devices 

>  Auditory 

> 

Light  Pen 

> 

Query  Language 

>  Color 

> 

Log-On 

> 

Security 

>  Command  Language 

> 

Manual 

> 

Tables 

>  Cursor 

> 

Menu 

> 

Touch  Screen 

>  Direct  Manipulation 

> 

Mouse 

> 

Trackball 

>  Function  Keys 

> 

Multiple  Users 

> 

Training  . 

>  Graphics 

> 

On-line  Guidance 

> 

Windows 

Move  to  a  FEATURE,  then  press  F4  to  see  its  definition  below: 


Next  to  each  feature 
type  Y  if  the  system  has 
that  feature  or  N  if  it 
does  not. 

Blank  also  equals  Y. 

Press  F2  to  save. 

Press  ESC  to  cancel. 
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To  complete  the  profile,  enter  a  Y  (if  the  feature  is  present  in  the  system  to  be  evaluated)  or 
N  (If  the  feature  is  not  present  in  the  system)  for  each  of  the  items  on  the  features  list.  Features 
marked  "Y"  will  have  their  associated  questions  included  in  the  questionnaire:  those  marked  'N* 
will  cause  questions  associated  with  those  features  omitted  from  the  questionnaire.  Note  that  a 
blank  entry  is  equivalent  to  a  "Y"  for  the  purposes  of  this  screen. 

If  you  are  unsure  of  the  meaning  of  a  particular  feature,  press  F4  for  a  definition.  That 
definition  will  remain  on  the  screen  until  you  ask  for  the  definition  of  another  feature.  When  you 
have  completed  your  responses  to  the  feature  list,  press  F2  to  save  and  continue,  or  ESC  to 
cancel  and  return  to  the  Main  Menu. 

Pressing  F2  will  save  the  system  profile  and  lead  you  to  the  Options  menu  shown  on  the 
following  page. 


OPTIONS  for  System  Profile:  ALPHA12 

Edit  Question  List 

Print  Question  List 
Print  System  Profile 
Return  to  the  Main  Menu 


Edit  questionnaire  by  deleting  items  for  this  profile 

The  questionnaire  at  this  point  contains  questions  included  by  virtue  of  the  system  profile 
provided  in  the  previous  screen.  You  may  now: 

1)  Edit  the  questionnaire  through  item-by-item  review. 

2)  Print  the  question  list  associated  with  the  current  system  profile. 
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3)  Print  the  system  profile  (i.e.,  the  collection  of  Ys  and  Ns  entered  in  the  previous 

screen). 

4)  Return  to  the  Main  Menu. 

Selection  of  Edit  Question  List  produces  the  following  screen: 

DELETE  SUE  QUESTIONS  FROM  SET:  ALPHA12 


1  of  122 


Item  ID  Question/Statement 

1  The  user  is  adequately  prompted  when  data  entry  is  required. 

Press  DEL  to  delete  question  from  this  set. 

Press  PgDn  to  view  next  question. 

Press  PgUp  to  view  previous  question. 

Press  F2  to  Save  Question  Set  and  return  to  Main  Menu. 

Press  ESC  to  cancel  all  deletions  and  return  to  Options  Menu. 

From  this  screen,  you  review  each  of  the  attribute  statements  included  in  the  questionnaire 
by  virtue  of  the  system  profile.  Delete  any  attribute  statement  that  you  wish  to  omit  from  your 
SUE  questionnaire.  Each  item  in  the  profile  set  will  appear  sequentially.  Once  attribute 
statements  are  deleted  and  the  questionnaire  is  saved  (i.e.,  by  pressing  F2),  these  items  are 
permanently  removed  from  the  question  set. 


To  Review  and  Edit  a  Question  List: 

Scroll  through  items  in  the  question  set  using  the  Page  Up/Page  Down  keys. 
To  Delete  the  item  on  the  screen,  press  DELETE. 

To  Save  the  Question  Set  after  deletions  have  been  made.  Press  F2. 

Press  ESC  to  Cancel  all  item  deletions  and  Return  to  the  Options  menu. 
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If  you  select  the  Print  Question  List  option,  all  items  belonging  to  your  question  set  will  be 
printed  out.  You  may  interrupt  printing  once  a  question  set  print  has  started  by  pressing  the  Up 
arrow  (or  any  other  key).  This  will  to  bring  up  the  message:  "Printing  interrupted  -  Press  C  to 
Cancel  and  R  to  Resume  .*  To  stop  printing,  Press  C. 


To  Print  the  System  Profile: 

Select  Print  System  Profile. 

Highlight  your  menu  choice  by  moving  arrow  keys  UP  or  DOWN,  or  click  on  option 
with  mouse. 

Press  ENTER  to  select  option. 


3.2  EDITING  A  QUESTION  SET  OR  PROFILE 


Existing  questionnaires  or  profiles  may  be  edited  by  selecting  Edit  Questionnaire  or  Profile 
from  the  Questionnaire  Construction  menu. 


I 

I  SOFTWARE  USABILITY  EVALUATOR 

VERSION  2.6 


MAIN  MENU 

Questionnaire  Construction 
Create  New  Profile 
Edit  Question  Set  or  Profile 
Questionnaire 
Custom  Notes 
System  Profile 
Cancel 


Use  arrow  keys  to  highlight  your  choice,  then  press  ENTER; 
or  use  the  mouse  to  click  on  your  choice. 
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Edit  a  Questionnaire  by  deleting  items  from  the  question  set. 

From  this  menu,  you  continue  the  questionnaire  construction  process  with  one  of  the 
following  options; 

Questionnaire  *  Make  changes  to  a  question  set  by  deleting  items. 

Custom  Notes  -  Write  customized  notes  to  accompany  individual  question?. 

System  Profile  -  Make  changes  to  the  Profile  Information  or  system  feature 

list  associated  with  a  system  profile 

3.2.1  Editing  a  Questionnaire 

Once  you  press  Questionnaire,  you  need  to  specify  the  system  profile  that  you  wish  to  edit. 
The  screen  shown  on  the  following  page  allows  you  to  select  the  system  profile. 


EDIT  QUESTIONNAIRE 


System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

To  edit  a  Questionnaire: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then  press  ENTER  or  click  on  the 
OK  button. 


This  screen  presents  the  list  of  system  names  (i.e.,  system  profiles)  that  have  already  been 
•created  and  are  available  to  edit.  While  only  six  are  visible  on  the  screen,  you  may  scroll  to 
view  additional  system  names. 


14 


You  edit  a  questionnaire  by  eliminating  individual  questions  from  the  questionnaire 
associated  with  an  existing  profile.  To  select  the  system  name  whose  questionnaire  you  wish 
to  edit  from  this  list,  highlight  the  desired  system  name,  then  press  ENTER  or  click  the  OK 
button.  Press  or  click  on  CANCEL  to  return  to  the  Main  Menu. 

Once  a  questionnaire  is  selected,  editing  by  deletion  is  performed  in  the  same  manner  as 
that  explained  on  pages  11  and  12  under  Edit  Questionnaire  in  the  Questionnaire  Construction 
path.  Pressing  F2  ends  questionnaire  editing  and  saves  the  edited  questionnaire.  Options  at 
this  point  are; 

Print  Question  List:  Prints  the  question  list,  as  described  on  page  11. 

Return  to  the  Main  Menu:  Presents  the  Main  Menu  screen. 


To  Edit  a  Questionnaire: 

Select  Questionnaire  from  the  Edit  Question  Set  or  Profile  menu. 
Review  and  delete  individual  questions  as  desired. 

Press  F2  to  save. 

Press  ESC  to  return  to  the  previous  menu  without  saving  edits. 


3.2.2  Customizing  Notes 

Notes  are  used  to  make  the  attribute  statements  more  understandable  to  the  respondent. 
You  may  use  the  notes  to  direct  the  respondent's  attention  to  a  particular  aspect  of  the  software 
interface,  or  you  may  use  notes  to  restate  the  attribute  statement  in  terms  specific  to  the 
system  being  studies.  For  example  a  note  might  be;  "For  this  questionnaire,  consider  Milstar 
terminal  'viewports'  mean  the  same  thing  as  the  term  'windows'.'  Selecting  the  Custom  Notes 
option  produces  the  following  screen: 
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EDIT  QUESTIONNAIRE  NOTES 


1  System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

1  Delta 

BHT 

10/11/93 

. 

OK 

CANCEL 

V  - 

To  edit  a  Questionnaires  Notes:  | 

Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name, 

then  press  ENTER  or  click  on  the 

1  OK  button. 

On  this  screen  you  select  the  system  profile  for  which  you  want  to  customize  the  notes. 
The  next  screen  displays  the  individual  attribute  statements  associated  with  the  system  profile, 
and  allows  the  input  of  up  to  approximately  3  lines  of  customized  notes  to  accompany  each 
attribute  statement.  This  screen  is  shown  on  the  following  page: 

Question/Statement  1  of  86 

The  user  is  adequately  prompted  when  data  entry  is  required. 


Type  in  the  text  of  the  note  to  be  shown  with  the  Question/Statement  shown: 
above: 

(DATA  ENTRY  AREA) 


Press  PGDN  to  view  the  next  question  and  note. 

Press  PGUP  to  view  the  previous  question  and  note. 

Press  F2  to  save  all  notes  and  Return  to  the  Main  Menu. 
Press  ESC  to  abandon  all  notes  and  return  to  the  Main  Menu. 
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From  this  screen  you  are  able  to  customize  the  notes  associated  with  SUE  items.  You 
type  in  notes  in  the  data  entry  area,  then  use  the  arrow  keys  to  move  to  the  next  SUE  question. 
After  completing  your  custom  notes,  save  by  pressing  F2. 


To  Customize  Notes: 

Select  the  desired  system  profile. 

Scroll  the  question  list,  adding  notes  as  desired. 

Press  F2  to  save. 

Press  ESC  to  return  to  the  previous  menu  without  saving  notes. 


3.2.3  Editing  a  System  Profile 


Selecting  System  Profile  from  the  Edit  Questionnaire  or  Profile  menu  allows  the  user  to  edit 
the  system  profile,  i.e.,  the  collection  of  Ys  and  Ns  made  in  response  to  the  list  of  potential 
system  features.  The  Edit  System  Profile  screen  appears  on  the  following  page: 


EDIT  SYSTEM  PROFILE 


I  System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

1  Test  1 

TW 

10/13/93 

1  Test  2 

MDB 

4/29/92 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

To  select  a  System  Profile  to  edit: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then  press  ENTER  or  click  on  the 
OK  button. 
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This  screen  presents  the  list  of  System  Profiles  that  have  already  been  created  and  are 
available  to  edit.  While  only  six  names  are  listed  on  the  screen,  you  may  scroll  through  the  list 
to  view  additional  profile  names  by  pressing  the  DOWN  arrow.  To  select  the  system  profile  you 
wish  to  edit  from  this  list,  highlight  the  desired  system  profile  name,  then  press  ENTER  or  click 
the  OK  button.  Press  or  click  on  CANCEL  to  return  to  the  Main  Menu. 

Once  a  system  profile  has  been  selected,  you  may  edit  the  profile  information  using  the 
screen  that  appears  below: 


EDIT  SYSTEM  PROFILE  INFORMATION 

SYSTEM  INFORMATION: 

System  Name:  ALPHA12 
Press  ESC  to  Cancel  and  Return  to  Main  Menu. 
Description:  Adv  Radar  Detect  Sys 
Author:  MDB 


Use  the  Up  and  Dn  arrow  keys  to  move  cursor  to  the  informatiori  you  want  to  edit. 
Use  the  Backspace  key  to  Delete;  then  Type  new  entry. 


Press  F2  to  Save  and  Continue. 

Press  ESC  to  Cancel  and  Return  to  the  Main  Menu. 


When  you  are  in, the  Edit  System  Profile  menu  you  may  want  to  edit  an  existing  profile  in 
one  of  two  ways  -  either  by  assigning  new  system  identifiers  (Editing  System  Profile 
Information)  arid/or  by  changing  the  system  features  that  you  have  designated  for  a  system 
profile  (Editing  System  Profiles  List).  The  first  of  these  edits  is  handled  with  the  screen  shown 
above.  Changes  are  made  by  moving  to  the  appropriate  field,  backspacing  to  delete  the 
appropriate  characters,  and  typing  in  the  desired  substitutions.  If  there  are  no  additional 
changes  to  be  made  in  the  system  profile  information,  press  F2  to  proceed  to  the  Edit  System 
Profile  screen,  shown  below: 
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Editing  System  Profile:  ALPHA12 
FEATURES: 


Alarms 

Y  Keyboard 

Y 

Peripheral  Devices 

Auditory 

N  Light  Pen 

Y 

Query  Language 

Color 

N  Log-On 

Y 

Security 

Command  Language 

N  Manual 

Y 

Tables 

Cursor 

Y  Menu 

Y 

Touch  Screen 

Direct  Manipulation 

Mouse 

Y 

Trackball 

Function  Keys 

Y  Multiple  Users 

Y 

Training 

Graphics 

N  On-line  Guidance 

Y 

Windows 

Move  to  a  FEATURE,  then  press  F4  to  see  its  definition  below: 


Next  to  each  feature 
type  Y  if  the  system 
has  that  feature  or 
N  if  it  does  not. 

Blank  also  equals  Y. 

Press  F2  to  save. 
Press  ESC  to  cancel. 


The  Editing  System  Profile  screen  functions  in  the  same  manner  as  the  Create  New 
System  Profile  screen  used  to  create  the  original  system  profile,  except  that  the  blanks  have 
been  replaced  with  the  Ys  and  Ns  provided  during  system  profile  creation.  To  change  those  Ys 
and  Ns,  position  the  cursor  alongside  the  response  to  be  changed,  use  the  backspace  key  to 
delete  the  Y  or  N,  then  type  in  Y  or  N  as  necessary.  When  all  desired  changes  have  been 
made,  press  F2  to  save  and  continue. 
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To  Edit  a  System  Profile: 

Select  the  desired  system  profile 

Make  desired  changes  in  the  system  profile  information 

Make  desired  changes  in  the  system  profile 

Press  F2  to  Save  and  Continue. 

Press  ESC  to  cancel  and  return  to  the  Main  MetuL _ 

Special  Note: 

Since  question  editing  (i.e.,  deletion  of  individual  questions)  and  customized  notes  are 
tied  to  specific  system  profiles,  any  changes  made  to  the  system  profile  will  cancel 
those  edits  and  customized  notes. 

After  editing  a  system  profile,  it  will  be  necessary  to  repeat  the  question  editing  and 
custom  notes  processes. 


After  you  Press  F2,  the  system  provides  a  warning  screen  to  remind  you  that  any  changes 
made  in  the  system  profile  will  require  you  to  re-edit  the  question  set  and  re-enter  customized 
notes.  Select  Save  and  Continue  to  acknowledge  this  warning  or  Cancel  Editing  to  abort  the 
system  profile  edit.  Selecting  Save  and  Continue  leads  to  the  Options  screen  shown  below. 


OPTIONS  for  System  Profile:  ALPHA12 

Edit  Questionnaire 

Print  Question  List 
Print  System  Profile 
Return  to  the  Main  Menu 


Edit  the  questionnaire  by  deleting  items  for  this  profile 

These  are  the  same  options  that  were  available  after  the  system  profile  was  first  created. 
From  the  Options  screen,  you  may  select  one  of  the  following  functions: 
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Edit  Question  List  -  Edit  questionnaire  by  deleting  items  for  this  profile. 

Print  Question  List  -  Print  the  question  list  defined  by  the  current  system  profile. 

Print  System  Profile  -  Print  the  system  profile  (i.e.,  the  collection  of  Ys  and  Ns). 

Return  to  the  Main  Menu. 

The  meaning  and  uses  of  these  options  are  discussed  on  pages  12  and  13. 

3.3  DELETING  A  PROFILE 

Profiles  are  deleted  by  selecting  the  Delete  Profile  option  from  the  Questionnaire 
Construction  menu.  Note  that  deleting  a  profile  will  delete  the  questionnaire  associated  with 
that  profile,  including  any  tailoring  (i.e.,  question  set  tailoring  and/or  customized  notes)  that  has 
been  done  on  the  question  set.  The  initial  Delete  Profile  screen  is  shown  below: 


DELETE  SYSTEM  PROFILE  AND  ASSOCIATED  QUESTIONNAIRE 


On  this  screen,  you  are  presented  with  a  scrollable  list  of  system  profile  names.  From  this 
list,  you  select  a  profile  to  delete.  Then,  on  the  following  screen,  you  will  be  asked  to  confirm 
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the  deletion.  (This  gives  you  the  chance  to  review  the  system  features  associated  with  the 
profile  to  be  sure  it  is  the  Profile  that  you  want  to  delete.) 

After  a  system  profile  is  selected  for  deletion,  the  system  information  and  feature  set 
belonging  to  that  profile  are  shown  to  give  you  another  opportunity  to  cancel  the  deletion 
command  if  you  determine  that  you  have  selected  the  wrong  system  profile  to  delete.  If  the 
system  profile  displayed  is  the  one  you  wish  to  delete,  press  DELETE  to  finalize  the  deletion. 
(Note:  A  delete  action  will  delete  both  the  system  profile  and  its  associated  questionnaire.)  To 
abort  the  deletion  and  return  to  the  Main  Menu,  Press  ESC. 


To  Delete  a  System  Profile: 

Highlight  the  desired  profile  name. 

Press  ENTER  or  click  on  OK  to  delete. 

Click  on  Cancel  or  press  ALT-C  to  cancel  the  delete  and  return  to  the  Main  Menu. 
At  the  confirmation  screen,  press  DELETE  to  confirm  deletion;  press  ESC  to  abort 
deletion.  _ _ 


Special  Note: 

Before  deleting  a  system  profile,  you  should  delete  any  data  files  (i.e.,  response  files) 
that  might  have  been  obtained  using  the  questionnaire  associated  with  that  profile. 
This  is  done  on  the  Utilities-Data  Management  path. 

If  you  do  not  first  delete  the  data  files,  these  files  will  be  unusable  and  impossible  to 
delete  after  the  system  profile  has  been  deleted.  Moreover,  even  though  they  are 
inaccessible  to  you,  the  data  file  names  will  remain  reserved  and  cannot  be  re-used. 
The  only  way  to  clear  this  condition  is  by  using  the  Paradox  application  program 
available  at  AFOTEC  headquarters. _ _ 


3.4  COPYING  A  PROFILE 

A  system  profile  may  be  copied  to  facilitate  the  creation  of  a  new  questionnaire  similar  to 
sn  existing  questionnaire.  Selecting  Copy  Profile  from  the  Questionnaire  Construction  menu 
produces  the  screen  shown  on  the  following  page: 

COPY  A  SYSTEM  PROFILE 
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System  Name 


Author 


Last  Modified  On 


ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

To  select  a  System  Profile  to  copy: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name;  then 
press  ENTER  or  click  on  the  OK  button. ^ ’ 


From  this  screen,  you  select  an  existing  system  profile  to  copy  from  the  list  of  system 
profiles,  by  highlighting  and  selecting  the  desired  profile.  The  screen  resulting  from  selecting  a 
system  profile  is  shown  below: 


System  Name:  ALPHA12  <-  You  MUST  Change  the  System  Name! 

Description:  F-1 6  Helmet  Display 
Author:  MDB 


Use  the  UP  and  DN  arrow  keys  to  move  cursor  to 
the  information  that  you  want  to  edit. 

Use  the  Backspace  key  to  Delete;  then  Type  new  entry. 


Press  F2  to  Save  and  Continue. 

Press  ESC  to  Cancel  and  Return  to  the  Main  Menu. 
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You  are  required  on  this  screen  to  assign  a  new  name  to  the  system  profile  you  are 
copying.  You  may  also  change  the  system  description  and/or  the  author  name,  but  these  are 
not  required  data  entry  fields. 


To  Copy  a  System  Profile: 

Select  the  System  Profile  you  wish  to  copy. 

Use  the  UP  and  DN  arrow  keys  to  move  cursor  to  the  information  you  want  to  edit. 
Use  backspace  key  to  delete;  then  type  new  entry. 

Press  F2  to  save  and  continue. 

Press  ESC  to  cancel  and  return  to  Main  Menu. 


If  you  Press  F2,  you  will  continue  on  to  the  Options  screen  below: 

Edit  Question  List  -  Edit  questionnaire  by  deleting  items  for  this  profile. 

Print  Question  List  -  Print  the  question  list  defined  by  the  current  system  profile. 

Print  System  Profile  •  Print  the  system  profile  (i.e.,  the  collection  of  Ys  and  Ns). 

Return  to  the  Main  Menu. 

These  are  the  same  options  that  appeared  after  the  original  system  profile  was  created. 
Their  meaning  and  use  are  discussed  on  pages  12  and  13. 
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SECTION  4 

PRINTING  A  QUESTIONNAIRE  FORM 


The  Print  Questionnaire  Form  option  which  is  selected  from  the  Main  Menu  prints  a 
questionnaire  that  has  been  tailored  by  system  profile  and  by  optional  question  editing.  The 
printed  questionnaire  is  suitable  for  administration  of  the  SUE  in  cases  where  computers  may 
not  be  available  for  administration  or  may  be  precluded  for  some  other  reason  (e.g.,  security 
considerations).  Selecting  Print  Questionnaire  Form  produces  the  following  screen: 


SELECT  QUESTIONNAIRE 


1  System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL  - 

^  ^  ^  , 

For  a  hard  copy  printout  of  a  Questionnaire:.  I 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name;  then 
press  ENTER  or  click  on  the  OK  button. _  . 


From  this  screen,  you  select  the  system  profile  to  generate  a  hard  copy  questionnaire. 
Once  you  have  selected  the  profile  name,  the  hard  copy  SUE  questionnaire  will  begin  printing. 
You  may  interrupt  printing  once  a  profile  print  has  started  by  pressing  the  UP  arrow  (or  any 
other  key).  This  will  to  bring  up  the  message:  "Printing  interrupted  -  Press  C  to  Cancel  and  R 
to  Resume."  To  stop  printing.  Press  C. 
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To  Generate  a  Printed  Questionnaire: 

Select  the  system  profile  from  which  to  generate.a  hard  copy  SUE  questionnaire: 
Press  ENTER  or  click  on  OK  to  select. 

After  you  have  completed  your  hard  copy  SUE  generation,  you  may  return  to  the 
Main  Menu  by  pressing  ESC,  and  Exit  the  System. _ ^ _ 


Special  Note: 

When,  SUE  prints  a  questionnaire,  it  first  prints  all  question  pages,  numbered  1-n 
Next,  it  prints  all  of  the  additional  information  (i.e..  Notes,  Definitipns,  &  Examples) 
associated  with  those  questions  on  pages  numbered  1  A-nA. 

For  administration,  these  pages  need  to  be  collated  so  that  page  1 A  faces  page  1 , 
page  2A  faces  page  2,  and  so  on. 

This  will  ensure  that  questions  are  properly  aligned  with  their  additional  information^ 
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SECTION  5 

ON-LINE  ADMINISTRATION 


The  On-line  Administration  option,  selected  from  the  Main  Menu,  leads  the  administrator 
through  the  selection  and  initialization  of  on-line  administration,  and  then  leads  the  respondent 
through  the  questionnaire  itself.  The  administration  process  begins  with  identification  of  the 
profile  name  of  the  questionnaire  to  be  administered,  selected  through  the  following  screen: 


ON-LINE  SOFTWARE  USABILITY  EVALUATOR 


System  Name 

Author 

Last  Modified  On  | 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Tests 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

I  To  select  a  Questionnaire  and  begin  collecting  data: 

Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name;  then 
press  ENTER  or  click  on  the  OK  button. 


Once  the  desired  questionnaire  has  been  selected,  on-line  software  usability  evaluation  is 
turned  over  to  the  system  evaluator,  or  "respondent,*  who  proceeds  through  one  of  the 
following  interactive  steps: 

Read  Instructions:  Provides  instructions  on  using  the  automated  SUE. 

Begin  the  Questionnaire:  Starts  administration  of  the  questionnaire. 
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Resume  the  Questionnaire:  Resumes  an  administration  session  that  has  been  previously 
started  and  interrupted. 

Return  to  Main  Menu. 

On-line  questionnaire  administration  begins  with  a  welcome  screen,  displaying  options 
available  to  the  respondent  as  shown  below: 


WELCOME 


Thank  you  for  participating  in  this  data 
collection  effort 

Press  Enter  to  read  the  instructions  or 
Arrow  to  another  choice  and  press  Enter 


Read  Instructions 

Begin  the  Questionnaire 
Resume  the  Questionnaire 
Return  to  Main  Menu 


Read  instructions  before  starting  the  questionnaire 

The  respondent  may  select  the  first  menu  item  on  this  screen  to  read  SUE  instructions.  Or, 
if  he  is  already  familiar  with  the  instructions,  he  may  go  directly  to  the  first  item  of  the  software 
usability  evaluation. '  A  third  option  from  this  screen  is  to  resume  to  an  evaluation  in  progress 
that  had  previously  been  interrupted. 


5.1  READ  INSTRUCTIONS 


The  first  page  of  respondent  instructions  is  shown  below: 


SOFTWARE  USABILITY  EVALUATION  INSTRUCTIONS 
-  INTRODUCTION 
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This  questionnaire  contains  a  series  of  questions  about  the  usability  of  the  software  in  the 
system  you  have  been  testing.  In  general,  "usability"  simply  means  how  easy  the  software  is  to 
use.  In  this  questionnaire,  usability  will  be  assessed  in  terms  of  the  presence  or  absence  of  a 
set  of  software  quality  features. 

Potential  quality  features  will  be  presented  to  you  one  at  a  time.  Your  task  is  to  indicate  if  the 
feature  described  is  always  present  or  not  always  present  in  the  software  being  evaluated. 
Responses  of  "DON'T  KNOW"  or  "NOT  APPLICABLE"  may  also  be  selected,  if  appropriate. 

PAGE  1  of  4 

Press  Page  Down  to  continue  with  the  instructions. 

Press  ESC  to  continue  with  the  questionnaire. 

The  respondent  is  presented  with  four  pages  of  instructions  about  how  to  take  the  on-line 
Software  Usability  Evaluator  Questionnaire.  The  instruction  pages  are  organized  as  follows: 

Page  1 .  Introduction 

Page  2.  Questionnaire  Layout 

Page  3.  How  to  Respond  to  the  Questionnaire 

Page  4.  How  to  Respond  to  the  Questionnaire 


5.2  BEGINNING  THE  QUESTIQNNAIRE 

Selecting  Begin  Questionnaire  begins  the  sequence  that  collects  identification  data  from 
the  respondent  and  then  administers  the  questionnaire. 

5.2.1  Entering  Respondent  Information 

In  the  first  step  of  questionnaire  administration,  the  respondent  identifies  himself  with  the 
following  screen: 
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RESPONDENT  INFORMATION 


Name:  (optional) 

ID  Number:  (numbers  only) 

Rank: 

Duty  Position: 

Years  of  Experience:  (optional) 


Enter  the  information  above  and  Press  F2  to  Continue. 
Press  F1  for  Help  with  data  entry  format. 


On  this  screen,  the  respondent  enters  information  identifying  him  and  his  duty  position. 
The  following  information  is  required:  ID  Number  (Enter  from  1  to  7  numbers;  no  spaces  or 
special  characters  allowed):  Rank  (1  to  5  alphanumeric  characters):  and  Duty  Position  (10  or 
fewer  characters).  Entries  for  Name  and  Years  of  Experience  are  optional.  If  the  respondent 
attempts  to  an  ID  Number  that  already  exists,  the  following  message  appears,  and  you  must 
enter  a  new  ID  number: 

Duplicate  Identification  Code 

Type  in  a  new  identification  Code:  _ 

Please  use  only  numbers. 

5.2.2  Questionnaire  Administration 

The  questionnaire  requires  several  types  of  responses,  as  described  in  the  following 
sections. 

5.2.2. 1  Frequency  Rating 

The  first  response  is  a  rating  of  the  frequency  with  which  a  given  usability  attribute  is  found 
in  the  software  being  evaluated.  The  screen  for  this  response  is  shown  below : 

***  Question  1  of  84  *** 

The  user  is  adequately  prompted  when  data  entry  is  required. 
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=====  THIS  FEATURE  IS  PRESENT:  ======= 

(o)  Always  ( )  Not  Always  ( )  Don't  Know  ( )  Not  Applicable 

Continue  Previous  Question  Interrupt  data  entry 


-  ADDITIONAL  INFORMATION  - 

NOTE:  This  question  addresses  the  points  in  time  during  which  data  entry 
is  required. 

DEFINITION:  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
EXAMPLE:  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 


The  respondent  is  required  to  assign  a  frequency  rating  (Always,  Not  Always,  Not 
Applicable,  Don't  Know)  for  the  usability  attribute  by  using  the  arrow  keys  to  move  to  the 
desired  response  and  pressing  ENTER.  In  considering  his  answer,  the  respondent  is 
encouraged  to  make  use  of  the  "Additional  Information"  (i.e.,  an  analyst-generated  NOTE 
relating  to  the  item,  a  DEFINITION  for  key  terms,  and  an  EXAMPLE  of  how  the  usability 
attribute  might  manifest  itself),  which  may  appear  at  the  bottom  of  the  screen. 

To  indicate  his  answer,  the  respondent  must  use  the  arrow  keys  or  mouse  to  select  one  of 
the  four  ratings,  then  press  ENTER  or  click  the  mouse  to  continue.  In  the  event  that  it  is 
necessary  to  interrupt  data  entry  and  save  completed  items,  the  respondent  or  test 
administrator  may  select  the  INTERRUPT  button  and  Press  ENTER  or  click  the  mouse.  This 
will  save  all  prior  responses  and  return  the  program  to  the  Main  Menu. 

5.2.2.2  Respondent-Generated  Examples 

When  the  respondent  indicates  that  a  usability  attribute  is  "Not  Always"  present,  he  is 
asked  to  provide  a  system-specific  example  of  a  problem  related  to  this  item.  The  respondent 
types  in  his  example  in  the  text  entry  box  which  appears  on  the  screen. 


*•*  Question  1  of  84  *** 

The  user  is  adequately  prompted  when  data  entry  is  required. 


Give  an  example  of  where  the  feature  is  lacking.  Press  F2  when  done. 
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_ ADDITIONAL  INFORMATION - 

NOTE:  This  question  addresses  the  points  in  time  during  which  data  entry  is  required. 

DEFINITION:  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
EXAMPLE:  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 


To  enter  his  example,  the  respondent  types  his  input,  up  to  a  maximum  of  approximately  4 
lines.  Any  alphanumeric  character  is  appropriate  for  this  input,  and  the  Backspace  key  may  be 
used  to  revise  an  entry.  When  his  input  is  complete,  the  respondent  presses  F2  to  save  and 
continue. 

5.2.2.3  Operational  Impact  Ratings 

After  he  has  entered  his  system-specific  example  of  where  the  usability  attribute  is 
believed  to  be  lacking,  the  respondent  is  asked  to  provide  a  rating  of  the  operational  impact  of 
the  usability  deficiency.  This  is  done  with  a  5-point  impact  rating  scale  that  appears  after  the 
respondent  completes  entry  of  his  example. 


•**  Question  1  of  84 

The  user  is  adequately  prompted  when  data  entry  is  required. 

=============  PLEASE  RATE  THE  IMPACT:  ========= 

0  Very  Low  ( )  Low  (o)  Moderate  ()High  ()  Very  High 

Continue  Previous  Question  EDIT  Example 


Example  Text  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwoooooocxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

================  ADDITIONAL  INFORMATION  =============== 

NOTE:  This  question  addresses  the  points  in  time  during  which  data  entry 
is  required. 

DEFINITION:  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
EXAMPLE:  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
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The  respondent  uses  the  Left  and  Right  Arrow  keys  to  move  across  the  rating  scale  and 
the  TAB  key  to  toggle  between  Continue,  Previous  Question,  and  EDIT  Example  options. 
Alternatively,  the  respondent  may  point  and  click  with  the  mouse  to  indicate  his  rating  and 
continue. 

The  "Continue*  option  is  the  default  (that  is,  by  pressing  ENTER  after  positioning  the  cursor 
on  the  desired  impact  rating,  you  will  continue  to  the  next  questionnaire  item).  The  respondent 
may  return  to  previous  items  (e.g.,  to  review  or  change  a  response,  or  add  examples  that  come 
to  mind  later)  by  pressing  the  Previous  Question  option  on  the  rating  scale.  This  moves  the 
respondent  back  to  the  previous  question,  where  he  may  change  or  add  to  an  item  example 
from  this  screen  by  tabbing  over  to  the  EDIT  Example  button,  and  then  pressing  ENTER. 

Once  the  impact  rating  is  selected  and  entered,  the  program  moves  to  the  next  attribute 
statement,  if  any.  If  the  respondent  has  just  answered  the  last  question,  the  program 
terminates  that  respondent's  session  and  returns  to  the  Main  Menu. 

5.2.2.4  Interrupting  and  Resuming  the  Questionnaire 

Selecting  Interrupt  data  entry  from  the  basic  question  screen  interrupts  the  questionnaire, 
saves  the  data,  and  returns  the  program  to  the  Main  Menu.  Where  this  has  been  done  by  one 
or  more  respondents,  the  questionnaire  may  be  resumed  from  the  point  of  ,  interruption  by 
selecting  Resume  the  Questionnaire  from  the  Begin  Questionnaire  menu.  The  software  then 
displays  a  list  of  interrupted  response  files,  listed  by  ID  number,  as  shown  below: 


RESUME  QUESTIONNAIRE 

Respondent  ID  List 

123 

1234 

33444 

34567 

45627 

8765023 


OK  CANCEL 


To  resume  the  questionnaire  from  where  you  left  it: 

Use  the  UP  and  DOWN  arrows  to  highlight  the  Respondent's  ID 
then  press  Enter  or  click  on  the  OK  button. 

Select  the  appropriate  respondent  ID  to  resume  an  interrupted  questionnaire.  The  program 
will  automatically  move  to  the  item  following  the  last  completed  item  of  the  interrupted 
questionnaire. 
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On  this  screen,  the  OK  command  button  is  the  default,  and  is  activated  through  a  keyboard 
gfgygp  l^ey  press,  once  the  cursor  is  positioned  on  the  desired  respondent  ID  field.  To  activate 
the  CANCEL  command,  click  it  with  the  mouse. 


Select  a  frequency  rating  ("Always,"  "Not  Always,"  "Don't  Know,"  or  "Not  Applicable" 
and  Continue 

If  "Always,"  "Don't  Know,"  or  "Not  Applicable,"  proceed  with  the  next  question 
If  "Not  Always,"  input  a  system-specific  example  of  where  the  attribute  is  lacking  and 
then  provide  a  rating  of  the  operational  impact  of  the  usabiljtydeficie^ _ 
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SECTION  6 

ANALYSIS  AND  REPORTING 

SOFTWARE  USABILITY  EVALUATOR 
VERSION  2.6 


MAIN  MENU 

Questionnaire  Construction 

> 

Print  Questionnaire  Form 

On-line  Administration 

Analysis  &  Reporting 

Utilities 

> 

Exit 

Use  arrow  keys  to  highlight  your  choice,  then  press  ENTER;  or  use  the  mouse  to  click  on  your 
choice. 


Analyze  the  collected  questionnaire  data  and  print  reports 

When  you  are  ready  to  perform  data  analyses  and  generate  reports  for  a  completed 
questionnaire  set,  you  select  the  Analysis  and  Reporting  option  from  the  Main  Menu.  However, 
prior  to  performing  an  analysis,  you  will  need  to  have  created  a  full  data  set  (combine  ail 
respondent  data  files  for  a  given  system  test)  or  a  subset  (combine  selected  data  files  for  a 
given  system  test).  You  do  this  through  the  Utilities  >  Data  Management  path  (See  Section  7). 


6.1  SELECTING  A  QUESTIONNAIRE  FOR  ANALYSIS 

Questionnaires  are  selected  for  analysis  using  the  screen  shown  below  on  the  following 
page: 
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SELECT  QUESTIONNAIRE  FOR  DATA  ANALYSIS 


System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

n  - 

- - - - - - - - - ^ - 1 

To  select  a  Questionnaire  for  analysis: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then  press  ENTER  or  click 
on  the  OK  button.  


From  this  screen,  you  select  a  questionnaire  for  data  analysis.  While  only  six 
questionnaires  are  visible  in  the  window  list,  you  may  scroll  to  view  the  other  questionnaires. 

6.2  DATA  SETS  AND  SUBSETS 

Analysis  may  be  performed  on  either  all  of  the  response  files  to  a  particular  questionnaire 
(i.e.,  "Full  Data  Set")  or  on  a  subset  of  the  data  (i.e.,  "Data  Subset"),  partitioned  by  the  analyst. 
The  following  screen  is  used  to  indicate  which  of  these  two  types  of  analysis  is  desired: 


DATA  FILE  OPTIONS  FOR  QUESTIONNAIRE:  ALPHA12 

Use  Full  Data  Set 

Use  Data  Subset 
Return  to  the  Main  Menu 


Perform  analyses  using  the  full  data  set 
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6.2.1  Full  Data  Sets 


A  full  data  set  is  made  up  of  all  respondent  files,  for  a  given  questionnaire.  Prior  to 
requesting  an  analysis  on  the  full  data  set,  it  is  necessary  to  use  the  Create  a  Full  Data  Set 
option  in  the  Utilities  >  Data  Management  menu  path  (see  Section  7.2.2).  Upon  execution  of 
the  Create  Full  Data  Set,  the  software  will  inform  you  how  many  files  were  included  in  the  full 
data  set. 

6.2.2  Data  Subsets 

A  data  subset  is  a  collection  of  respondent  files  selected  for  analysis.  The  subset  may  be 
based  on  duty  position  (e.g.,  analyze  results  for  all  Gunners)  on  rank  (e.g.,  analyze  results  for 
all  Capts)  or  on  a  user-defined  subset  of  files  selected  by  the  analyst  (e.g.,  analyze  results  for 
respondent  ID  numbers  1,  3,  4,  and  7).  The  type  of. subset  to  analyze  is  selected  from  the 
Analysis  &  Reports  menu  using  the  screen  shown  below: 


DATA  FILE  OPTIONS  FOR  QUESTIONNAIRE:  APLHA12 
Use  Full  Data  Set 
Use  Subset  of  Data 
Duty  Position 
Rank 

User  Defined  


Perform  analyses  using  one  Duty  position 

For  subsets  of  duty  position  or  rank,  SUE  will  provide  a  list  of  available  duty  positions  or  rank. 
You  select  which  of  these  you  wish  to  use  for  the  desired  subset.  User  defined  subsets  require 
that  the  analyst  select  the  desired  respondent  files  individually  by  ID  number.  This  must  be 
done  using  the  Create  a  Specific  Data  Subset  option  in  the  Utilities  >  Data  Management  menu 
path  (see  Section  7.2.3)  prior  to  selecting  User  Defined  from  the  above  screen. 

6.3  SELECTING  AN  ANALYSIS  OPTION  , 

If  full  or  subset  data  files  have  not  been  previously  created  using  the  Utilities  >  Data 
Management  option,  appropriate  error  messages  will  be  generated.  If  such  files  have  been 
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created,  the  following  screen  will  appear  after  the  selection  of  either  the  Use  Full  Data  Set  or 
Use  Data  Subset  options: 

ANALYSIS  OPTIONS  FOR  SYSTEM  PROFILE:  ALPHA12 

NOT  ALWAYS  Responses  Only 

All  Responses 
Return  to  Main  Menu 


Analyze  the  questions  with  NOT  ALWAYS  Responses 


This  screen  presents  two  different  options  for  displaying  results: 

1)  NOT  ALWAYS  Responses  Only:  Processes  data  only  for  attribute  statements 
receiving  one  or  more  responses  of  "Not  Always."  This  option  will  process  only  the  "Not 
Always"  responses  and  impact  ratings.  Comments  will  not  be  displayed. 

2)  All  Responses:  Displays  data  for  all  attribute  statements  in  a  questionnaire, 
including  those  receiving  responses  of  "Always,"  "Not  Applicable,"  and  "Don't  Know."  This 
option  will  display  the  frequency  rating,  comment,  and  impact  rating  for  every  respondent  and 
each  question. 

Where  NOT  ALWAYS  Responses  Only  is  selected,  the  analyst  must  also  select  the 
manner  in  which  results  are  to  be  grouped.  The  choices  are: 


1)  Responses  Grouped  by  Number  of  Not  Always  Responses:  Attributes  are 

grouped  by  number  of  "Not  Always"  responses,  with  the  attributes  receiving  the  greatest 
number  presented  first.  Attribute  statements  with  the  highest  number  of  "Not  Always" 
responses  are  grouped  and  presented  at  the  top  of  the  listing,  followed  by  groups  of  attribute 
statements  with  successively  smaller  numbers  of  "Not  Always"  responses.  Each  attribute 


38 


b)  Interactive  Control.  User  actions  and  the  resultant  computer  responses  that 
control  the  activities  of  the  system. 

c)  Data  Display.  Software-programmable  output  of  data  presented  to  the  user. 

d)  User  Guidance.  On-line  displays  and  hard-copy  materials  which  provide 
instruction  and/or  system  information  to  the  user. 

e)  Mission  Performance.  Special-purpose  tasks  that  support  the  specific 
functions  for  which  the  system  has  been  designed. 

Within  a  given  function,  results  are  ordered  by  the  sum  of  the  impact  ratings  across 
respondents. 

After  selecting  a  data  reporting  format,  you  will  be  presented  another  screen  with  the 
Analysis  Report  Options  (i.e.,  to  specify  where  to  send  summary  analyses).  These  options  are 
discussed  in  the  following  section. 

6.4  ANALYSIS  AND  REPORT  OPTIONS 

The  Analysis  Report  Options  screen  is  shown  below: 


ANALYSIS  REPORT  OPTIONS: 

View  the  Report  on  the  Screen 

Print  the  Report  to  the  Printer 
Print  the  Report  to  a  File 
Return  to  Main  Menu 


Scroll  through  report  on  the  screen 
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statement  is  accompanied  by  a  frequency  distribution  showing  the  frequency  with  which  that 
attribute  statement  received  each  of  the  five  impact  ratings.  Within  groups  receiving  equal 
numbers  of  "Not  Always"  responses,  attribute  statements  are  ordered  by  the  total  impact  rating 
for  that  attribute  statement  across  respondents,  using  the  scale:  Very  Low=1,  Low=2, 
Moderate=3,  High=4,  and  Very  High=5. 

2)  Results  Grouped  by  Usability  Characteristic:  Groups  each  attribute  into  one  of  5 
system  characteristics,  which  describe  a  general  usability  trait.  These  characteristics  are. 

a)  Simplicity.  Minimization  of  mental  and/or  physical  workload  by  designing 
toward  user  capabilities,  limitations,  and  expectations. 

b)  Consistency.  Similarity  and  predictability  of  form  and  function  across 
different,  but  related  elements  of  the  interface. 

c)  Descriptiveness.  -Completeness  and  understandability  of  labels,  prompts, 
instructions,  and  messages. 

d)  Responsiveness.  Speed  and  precision  with  which  the  software  responds  to 
user  actions,  and  provides  the  user  with  a  feeling  of  "control"  over  the  software. 

e)  Error  Abatement.  Provisions  for  avoiding  and  correcting  user  errors,  and  for 
mitigating  the  impact  of  errors  that  have  occurred. 

Within  a  given  characteristic,  results  are  ordered  by  the  sum  of  the  impact  ratings  across 
respondents. 

3)  Results  Grouped  by  System  Function:  Groups  each  attribute  into  one  of  5  system 
functions,  which  describe  general  elements  of  the  software  interface  and  mission.  These 
functions  are: 

a)  Data  Entry.  Manual  input  of  information,  including  alphanumeric,  positional, 
and  graphic  data,  and  computer  responses  to  such  inputs. 
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These  Analysis  Report  Options  define  the  destination  of  your  analysis  file.  They  are  as  follows: 

a.  View  Report  on  the  Screen  -  Scroll  through  generated  report  on  the  screen.  To  exit 
the  on  screen  report,  click  on  Cancel  or  on  the  small  solid  box  in  the  upper  left  corner  of  the 
screen. 

If  you  select  the  View  Report  on  the  Screen  option,  you  see  the  report  on  the  screen  with  3 
pull  down  menu  choices: 

Goto  Search  Cancel 

These  menu  choices  and  their  options  are  described  below: 

Goto 

Page:  Move  to  a  specified  page  number  of  the  report. 

End  of  Report:  Move  to  the  end  of  the  report. 

Search 

Find:  Search  for  the  first  occurrence  of  a  specific  value  in  the  report. 

Next:  Search  for  the  next  occurrence  of  a  specific  value. 

Cancel 

No:  Continue  screen  viewing  of  report. 

Yes:  End  screen  viewing  of  report. 

b.  Print  Report  to  the  Printer  -  Send  report  output  to  the  printer. 

c.  Print  Report  to  a  File  -  Save  the  report  results  as  an  ASCII  file. 

If  you  select  the  Print  the  Report  to  a  File  option,  you  will  need  to  assign  an  output  file 
name.  The  following  form  will  appear  for  you  to  enter  an  output  file  name. 


Output  File  Name 
Enter  file  name  for  print  output: 
OK  CANCEL 
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SECTION  7 
UTILITIES 


The  Utilities  option,  which  is  selected  from  the  Main  Menu,  provides  a  variety  of  program 
and  data  management  capabilities.  The  Utilities  screen  is  shown  below: 


Software  Usability  Evaluator 
Version  2.6 

MAIN  MENU 

System  Profile 
Hard  Copy  Administration 
On-line  Administration 
Analysis  and  Reports 
Utilities 

Custom  Notes 

Enter  Data  from  Hard  Copy 

Data  Management 

Diskette  Setup 

Printer  Setup 

Cancel 


Edit  the  Notes  associated  with  each  Question 


The  following  utilities  and  data  management  functions  are  available  from  this  screen: 

Enter  Data  from  Hard  Copy  -  Enter  respondent  data  from  hard-copy  into  on-line 
SUE. 

Data  Management  -  Manage  data  files  and  sets  of  data. 

Diskette  Setup  -  Copies  questionnaire  files  from  a  host  computer  to  floppy  diskette. 
Transfer  Data  Files  -  Transfers  data  files  from  computer  to  diskette  and  from  diskette 
to  computer. 

Printer  Setup  -  Select  the  type  of  printer  for  hard-copy  questionnaires  and  reports. 
Cancel  -  Cancel  and  return  to  the  Main  Menu. 
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7.1  ENTERING  DATA  FROM  HARD  COPY 


This  utility  allows  you  to  take  data  that  were  obtained  through  hard  copy  administration  and 
enter  them  into  the  computer  for  processing  and  report  generation.  Once  entered,  data  input 
with  the  Enter  Data  From  Hard  Copy  utility  are  equivalent  to  data  that  are  entered  directly  into 
the  computer  via  on-line  administration.  The  screen  produced  by  this  selection  is  shown  below: 


SELECT  QUESTIONNAIRE  FOR  HARD  COPY  INPUT 


System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Tests 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

To  select  a  Questionnaire  for  hard  copy  data  entry: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then  press  ENTER  or  click 
on  the  OK  button.  . 


Once  you  have  selected  the  system  profile  for  which  you  are  entering  hard  copy  data,  the 
respondent  information  screen  comes  up,  and  you  are  required  to  enter  respondent  data.  This 
is  done  in  the  same  manner  as  if  you  were  the  respondent  completing  the  questionnaire  on-line. 

RESPONDENT  INFORMATION 

Name:  (optional) 

ID  Number:  (numbers  only) 

Rank: 

Duty  Position: 

Years  of  Experience:  (optional) 
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Press  F2  to  Continue. 

Press  F1  for  HELP  with  data  entry  format. 

Once  you  have  entered  the  required  respondent  information,  you  will  enter  the  respondents 
ratings  just  as  if  you  were  taking  an  on-line  SUE.  The  instructions  relating  to  taking  an  on-line 
Software  Usability  Evaluation  apply  here.  (See  Section  D  of  this  Guide.) 

After  the  hard  copy  data  entry  is  complete  for  the  first  respondent,  the  following  screen  will 
appear: 

OPTIONS  for  Hard  Copy  Data  Entry: 

Input  Data 

Select  New  System  Profile 

Return  to  Main  Menu 

Once  you  have  completed  the  data  entry  for  one  respondent  in  a  given  test,  you  may  either 
continue  entering  data  from  the  same  system  usability  test  (Input  Data),  enter  hard  copy  data 
for  a  different  system  test  (Select  New  System  Profile),  or  return  to  the  Main  Menu. 


7.2  DATA  MANAGEMENT 

The  Data  Management  option  allows  you  to  perform  a  variety  of  data  and  file 
manipulations.  The  screen  is  shown  below: 

Software  Usability  Evaluator 
Version  2.6 

MAIN  MENU 

Questionnaire  Construction 
Print  Questionnaire  Form 
Qn-line  Administration 
Analysis  and  Reports 
Utilities 

Enter  Data  from  Hard  Copy 
Data  Management 

Verify  Respondent  Data 
Create  a  Full  Data  Set 
Create  a  Specific  Data  Subset 
Delete  a  Respondent  File 
Cancel 
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Check  the  Respondent’s  input  for  Duty  Position  and  Rank. 

This  screen  presents  you  with  all  of  the  data  management  functions.  They  are; 

Verify  Respondent  Data  -  Check  respondent  duty  position  and  rank. 

Create  a  Full  Data  Set  -  Gather  all  response  files  for  a  given  questionnaire. 

Create  a  Specific  Data  Subset  -  Gather  selected  response  files  for  a  given  questionnaire. 

Delete  a  Respondent  File  -  Exit  Utilities  and  return  to  the  Main  Menu. 

Cancel  -  Check  respondent  duty  position  and  rank. 

7.2.1  Verifying  Respondent  Data 

Sometimes,  it  is  desirable  to  be  able  to  review  a  respondent's  description  of  his  duty 
position  and  rank  to  be  certain  that  the  respondent  is  consistent  in  terminology  with  others 
taking  the  questionnaire.  This  is  particularly  true  when  the  analyst  wishes  to  partition  results  by 
one  of  these  variables  (e.g.,  to  analyze  results  of  all  Gunners,  Pilots,  etc.).  The  Verify 
Respondent  Data  screen  is  shown  below: 


SELECT  QUESTIONNAIRE 


System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

OK 

CANCEL 

To  select  a  Questionnaire: 

Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then 
press  ENTER  or  click  on  the  OK  button. 

Selecting  a  questionnaire  presents  a  list  of  respondent  flies  for  that  questionnaire,  as  shown 
below: 
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VERIFICATION  OF  RESPONDENTS  DATA  FOR  SYSTEM  PROFILE:  ALPHA12 


ID 

1 

3 


NAME 

RANK 

DUTY  POSITION 

RESPONDENT  1 

SGT 

GUNNER 

RESPONDENT  3 

CPT 

PILOT 

OK 

CANCEL 

Selecting  one  of  the  respondents  produces  the  screen  shown  below: 


VERIFICATION  OF  RESPONDENT  INFORMATION 

ID  Number:  1  Name:  RESPONDENT  1 

Rank:  SGT 

Duty  Position:  GUNNER 


n 


Edit  the  Rank  and  Duty  Position  information  above. 
Press  F2  to  Save  and  Continue.  Press  ESC  to  Cancel. 


7.2.2  Creating  a  Full  Data  Set 

Prior  to  executing  analysis  and  report  options,  it  is  necessary  to  indicate  whether  you  wish 
to  perform  the  analysis  on  all  respondent  files  for  a  given  questionnaire  or  only  on  selected  files 
for  that  questionnaire.  The  Create  Full  Data  Set  option  is  used  to  select  all  respondent  files  for 
a  questionnaire.  The  screen  is  shown  below: 
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CREATE  A  DATA  SET  FOR  DATA  ANALYSIS 


Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

•  MDB 

4/29/92 

Test  3 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

To  select  a  Questionnaire  for  analysis: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then 
press  ENTER  or  click  on  the  OK  button. 


After  you  select  a  questionnaire,  the  system  responds  with  the  following  screen,  which 
shows  the  questionnaire  selected  and  the  number  of  respondent  files  found  for  that 
questionnaire. 


2  response  data  file(s)  found  for 
Questionnaire:  ALPHA12 


Full  Data  File  will  now  be  created. 


PRESS  ANY  KEY  TO  CONTINUE 


The  full  data  set  is  now  created.  Pressing  any  key  will  exit  and  return  the  program  to  the  Main 
Menu. 
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7.2.3  Creating  a  User-Defined  Data  Set 


If  you  wish  to  perform  a  selective  analysis  of  respondent  files,  you  may  partition  files  by 
selecting  the  individual  files  you  wish  to  analyze.  This  is  performed  with  the  Create  a  Specific 
Data  Set  option,  whose  initial  screen  is  shown  below: 


CREATE  A  DATA  SET  FOR  DATA  ANALYSIS 


Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

Tests 

MDB 

4/29/92 

Test  4 

MDB 

4/29/92 

Delta 

BHT 

10/11/93 

OK 

CANCEL 

!■  - 

■ 

To  select  a  Questionnaire  for  analysis: 

Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then 

1  press  ENTER  or  click  on  the  OK  button. 

- - - 1 
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Selecting  a  questionnaire  presents  a  list  of  respondent  files  for  that  questionnaire,  as  shown 
below: 


CREATE  A  DATA  SUBSET  FROM  THE  RESPONDENT  IDs 


List  of  Data  Files  for  System  Profile:  ALPHA12 


123 

1234 

3344 

345 

12 


Available  IDs:  IDs  to  be  Used: 

ADD» 


OK  CANCEL 


To  select  Respondent  Files  for  the  subset  data  file: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  IDs.  Then  click  on  the  ADD  button.  When 
the  new  list  is  complete,  press  Enter  or  click  on  the  OK  button. 


To  create  any  subset  of  the  available  respondent  files,  highlight  the  IDs,  one  at  a  time,  and 
click  on  the  ADD  button.  The  selected  ID  will  move  from  the  left  list  to  the  list  on  the  right.  The 
file  names  can  only  be  added  to  the  list;  if  you  add  one  by  mistake,  you  must  click  on  the 
CANCEL  button  and  start  over.  When  the  list  is  complete,  click  on  the  OK  button  and  the 
subset  will  be  created.  The  subset  can  be  used  under  the  analysis  and  reports  option  of  the 
Main  Menu. 
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7.2.4  Deleting  a  Respondent  File 


This  option  is  selected  to  delete  the  response  files  of  one  or  more  respondent  to  a  given 
questionnaire.  The  first  screen  of  this  option  requires  you  to  identify  the  questionnaire  to  which 
the  to-be-deleted  files  belong.  This  screen  is  shown  below: 


DELETE  RESPONDENT  FILE  FOR  SPECIFIC  QUESTIONNAIRE 


System  Name 

ALPHA12 
Test  1 
Test  2 
Test  3 
Test  4 
Delta 


Author 


BHT 

TW 

MDB 

MDB 

MDB 

BHT 


Last  Modified  On 

9/01/93 

10/13/93 

4/29/92 

4/29/92 

4/29/92 

10/11/93 

CANCEL 


To  select  a  Respondent's  file  for  a  particular  Questionnaire: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then  press  ENTER  or  click 
on  the  OK  button. 


The  next  screen  displays  the  available  respondent  files,  listed  by  respondent  ID,  for  the 
selected  questionnaire: 


DELETE  RESPONDENT  FILE 

Respondent  ID  List 
1 
2 

DELETECANCEL 
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If  you  select  DELETE,  the  software  will  ask  you  to  confirm  the  deletion  as  follows: 


7.3  DISKETTE  SETUP 

The  Diskette  Setup  option  allows  you  to  copy  to  diskette  all  the  files  necessary  to 
administer  a  particular  questionnaire.  This  option  assumes  that  the  target  computer  already 
possesses  a  copy  of  Paradox  Runtime  and  the  basic  SUE  software.  Thus  the  Diskette  Setup 
option  is  simply  designed  to  allow  you  to  place  a  copy  of  an  assembled,  edited  questionnaire  on 
one  or  more  remote  computers.  The  screen  is  shown  below; 
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CREATE  A  DISKETTE  FOR  FIELD  ADMINISTRATION 
DISKETTE  DRIVE  ° 

What  is  the  destination  drive 
for  the  files? 

(*)  Drive  A 
( )  Drive  B 

OK  Cancel 


NOTE; 

This  action  copies  only  the  files  associated  with  the  selected  Questionnaire.  The  SUE  software 
and  Runtime  Paradox  will  not  be  copied;  they  must  already  be  installed  on  the  field  computer 
system.  Instruct  the  user  to  copy  this  diskette  into  their  SUE  directory. 


The  next  step  in  this  process  is  to  identify  the  questionnaire  you  wish  to  copy,  using  the 
screen  below: 


SELECT  QUESTIONNAIRE 


System  Name 

Author 

Last  Modified  On 

ALPHA12 

BHT 

9/01/93 

Test  1 

TW 

10/13/93 

Test  2 

MDB 

4/29/92 

OK 

CANCEL 

i  n 

To  select  a  Respondent's  file  for  a  particular  Questionnaire: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then  press  ENTER  or  click 
nnJhe^QK^buttom^^^^^ _ 

Make  sure  there  is  a  blank  diskette  in  the  drive  selected.  After  OK  is  selected,  the  software  will 
copy  to  that  diskette  all  files  required  for  the  questionnaire  you  identified.  Following  this,  the 
program  will  return  to  the  Main  Menu. 
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7.4  TRANSFERRING  DATA  FILES 


The  Transfer  Data  Files  option  allows  you  to  copy  to  diskette  respondent  files  that  may 
have  been  collected  on  one  or  more  computers.  These  files,  which  are  called  field  data  files 
may  then  be  transferred  back  to  a  central  computer  for  collective  analysis.  The  Transfer  Data 
Files  option  provides  for  both  of  these  transfers  (i.e.,  from  distributed  computers  to  diskettes 
and  from  diskettes  to  a  centralized  computer).  The  menu  selections  are  shown  below: 

Software  Usability  Evaluator 
Version  2.6 


MAIN  MENU 

Questionnaire  Construction 
Print  Questionnaire  Form 
On-line  Administration 
Analysis  and  Reports 

Utilities 

Enter  Data  from  Hard  Copy 
Data  Management 

Diskette  Setup 

Transfer  Field  Data  Files  to  Diskette 
T ransfer  Diskette  Data  to  Computer 
Cancel 


Check  the  Respondent's  input  for  Duty  Position  and  Rank. 

Selecting  the  Transfer  Field  Data  Files  to  Diskette  option  indicates  that  you  wish  to  transfer 
respondent  files  from  a  field  (i.e.,  "distributed'')  computer  to  diskette.  After  selecting  this  option, 
you  are  required  to  indicate  the  destination  drive  for  the  files.  Additionally,  since  the 
questionnaire  file  is  required  for  data  analysis,  you  are  required  to  indicate  whether  or  not  the 
analysis  ("centralized")  computer  already  possesses  that  file.  If  it  does  have  the  file,  it  saves 
diskette  space  to  omit  the  questionnaire  file  from  the  transfer  process.  If  the  analysis  computer 
does  not  contain  the  questionnaire  file,  the  transfer  will  include  a  copy  of  that  file.  The  screen 
for  these  selections  is  shown  below: 
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After  you  have  pressed  Enter  or  OK,  the  system  will  prompt  you  to  make  sure  you  have 
placed  a  formatted  diskette  in  the  target  drive.  Note:  If  this  is  not  done,  the  program  will  abort 
and  return  you  to  the  C/>  prompt;  to  recover,  type  SUE  to  re-initialize  the  program. 


The  last  step  in  the  data  transfer  process  is  to  indicate  which  questionnaire  respondent  files 
you  wish  to  transfer.  This  is  done  with  the  following  screen: 


SELECT  QUESTIONNAIRE 


ALPHA12  BHT  9/01/93 

Test  1  TW  10/13/93 

Test  2  MDB  4/29/92 


ALPHA12  BHT  9/01/93 

Test  1  TW  10/13/93 

Test  2  MDB  4/29/92 

OK  CANCEL 
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To  select  a  Questionnaire  for  data  transfer: 


Use  the  UP  and  DOWN  arrows  to  highlight  the  system  name,  then  press  ENTER  or  click  | 
on  the  OK  button.  I 


When  you  have  selected  the  desired  questionnaire,  the  respondent  files  and  the  questionnaire 
file  (if  this  option  was  chosen  in  the  previous  screen)  will  be  copied  to  diskette  and  the  program 
will  return  to  the  Main  Menu. 

Selecting  the  Transfer  Diskette  Data  to  Computer  option  from  the  Transfer  Data  Files 
submenu  performs  the  reverse  operation  to  that  described  above.  Here,  files  that  have  been 
copied  to  diskette  via  the  Transfer  Field  Data  Files  to  Diskette  option  are  placed  into  the 
centralized  computer  for  analysis  and  report  generation.  The  screen  for  this  procedure  is 
shown  below: 

DISKETTE  DRIVE 

What  is  the  source  drive 
for  the  files? 

ODriveA 
( )  Drive  B 

OK  Cancel 


After  you  have  pressed  Enter  or  OK,  the  system  will  prompt  you  to  make  sure  you  have 
placed  a  formatted  diskette  in  the  target  drive.  Note:  If  this  is  not  done,  the  program  will  abort 
and  return  you  to  the  C/>  prompt;  to  recover,  type  SUE  to  re-initialize  the  program. 

Unlike  the  sequence  for  transferring  files  to  diskette,  the  transfer  of  diskette  files  to 
computer  does  not  require  you  to  indicate  the  questionnaire  from  which  to  draw  files.  It  is 
assumed  that  the  diskette  possesses  files  from  only  one  questionnaire.  Pressing  Enter  or  OK 
in  response  to  the  confirmation  screen  described  above  will  start  the  transfer  from  disk  to 
computer.  If  the  computer  already  contains  respondent  or  questionnaire  fijes  with  the  same 
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name,  you  will  be  queried  as  to  whether  you  wish  to  skip  the  transfer  of  the  duplicate  file  or 
overwrite  the  existing  file  on  the  computer. 

7.5  PRINTER  SETUP 

The  Printer  Setup  option  allows  you  to  load  the  printer  driver  software  appropriate  to  your 
printer.  In  SUE,  the  choices  are:  (1)  laser  or  dot  matrix  printer;  and  (2)  Postscript  printer. 
Selecting  the  Printer  Setup  option  from  the  Utilities  submenu  produces  the  screen  shown 
below: 


SELECT  PRINTER 

What  type  of  printer  will  be 
.used  the  reports? 

(*)  Laser  printer  or  dot  matrix 
( )  Postscript  printer 

OK  Cancel 


Use  the  cursor  keys  or  mouse  to  make  your  selection,  then  press  Enter  or  OK  to  activate 
the  desired  printer  driver.  Note  that  Laser  printer  or  dot  matrix  is  the  default  driver.  No  printer 
selection  is  ever  required  if  you  intend  to  use  these  printers.  However,  if  you  are  .using  a 
Postscript,  printer,  the  Postscript  printer  driver  will  need  to  be  selected  each  time  you  use  the 
SUE. 
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